Integers can be hexadecimal, decimal or octal. An optional case-insensitive
suffix specifies their size: HH: 8 bit, H/S: 16 bit, L: 32 bit, LL: 64 bit
An optional U suffix makes a number unsigned. Ordinary 0x hex numbers are
always treated as unsigned. +0x or -0x hex numbers are treated as signed
unless they have a U suffix. Unsigned integers cannot be larger than 2^64-1.
If n is an unsigned integer then -n is also a valid unsigned integer as in C.
Signed integers must fall into the [-2^63, 2^63-1] range or a correspondingly
smaller range when a suffix specifies a smaller type. Out of range signed
numbers trigger a warning.
Ordinary 0x hex numbers with n hex digits (counting leading zeros) use
the smallest size of 1, 2, 4 and 8 bytes that can accommodate any n-digit hex
number. If a suffix specifies a size explicitly the corresponding number of
least significant bytes are written. Otherwise, signed and unsigned integers
alike occupy the smallest of 1, 2, 4, or 8 bytes needed to accommodate them
in their respective representation.
Using strtoll() can only return numbers in the range [-2^63, 2^63-1]. This
means that 0xffffFFFFffffFFFF (2^64-1) will be out of range and is written as
max LL. Actually, every 64-bit number with high-bit set will wrongly be
written as max LL.
This commit uses strtoull() instead to fix this, and checks for unsiged out-
of-range error. strtoull() also has the neat benefit that input with a minus
sign is treated like C unsigned numbers, ie, -u is also a valid unsigned
number if only u is one. In case the input is meant to be treated as signed,
it is therefore still OK to use strtoull() in the first instance only that in
this case a second check against the range of the signed domain is necessary.
In certain situations (CRC failure, device locked), that JTAG3
read functions need to return an indication to the caller that
it is OK to proceed, and allow erasing the device anyway.
Historically, the JTAG3 code passed the respective protocol
errors directly (and unexplained) up to the caller, leaving
the decision to the caller how to handle the situation.
Replace that by a more common return value API. New code should
prefer this API instead of any hardcoded return values.