Using strtoll() can only return numbers in the range [-2^63, 2^63-1]. This
means that 0xffffFFFFffffFFFF (2^64-1) will be out of range and is written as
max LL. Actually, every 64-bit number with high-bit set will wrongly be
written as max LL.
This commit uses strtoull() instead to fix this, and checks for unsiged out-
of-range error. strtoull() also has the neat benefit that input with a minus
sign is treated like C unsigned numbers, ie, -u is also a valid unsigned
number if only u is one. In case the input is meant to be treated as signed,
it is therefore still OK to use strtoull() in the first instance only that in
this case a second check against the range of the signed domain is necessary.
The paged read in pickit2.c has two errors:
- It drops load extended address commands unless a paged read happens at a
64k byte boundary; this is invalid when reading files with holes
- It wrongly assumed that flash memory is byte addressed
The fix is to carry out a load extended address command, if needed, at the
beginning of each paged flash read with the correct word address. Although the
pickit2_paged_load() has independent parameters page_size, addr and n_bytes,
AVRDUDE only ever calls paged read/write functions with page_size and n_bytes
both set to mem->page_size and addr aligned with a page boundary. Therefore, it
is sufficient to set the load extended address at the beginning of each page
read.
In certain situations (CRC failure, device locked), that JTAG3
read functions need to return an indication to the caller that
it is OK to proceed, and allow erasing the device anyway.
Historically, the JTAG3 code passed the respective protocol
errors directly (and unexplained) up to the caller, leaving
the decision to the caller how to handle the situation.
Replace that by a more common return value API. New code should
prefer this API instead of any hardcoded return values.