mem: Do not check unsigned values for negative size