Faculty of Information Technology
Software Engineering Group

Opened 4 years ago

Last modified 4 years ago

#290 new defect

binary and hex inspector extensions for LimitedPrecisionReal are wrongly padded

Reported by: nicolas.cellier.aka.nice@… Owned by:
Priority: major Milestone:
Component: default Keywords: Float inspect
Cc: Also affects CVS HEAD (eXept version): no

Description (last modified by nicolas.cellier.aka.nice@…)

inspectorExtraAttributes is using paddedTo:with: instead of leftPaddedTo:with:

It is demonstrated below with ShortFloat? (because examples are shorter).

(1.0 asShortFloat inspect) gives:
hex 3F800000 (correct)
binary 11111100 10000000 00000000 00000000
should be 00111111 10000000 00000000 00000000

(1.0+(1<<13) reciprocal + (1<<21) reciprocal) asShortFloat inspect
all indexed vars:
1 : 4
2 : 4
3 : 128
4 : 63
binary 11111100 10000000 10000000 10000000
should be 00111111 10000000 00000100 00000100
hex 3F804040
should be 3F800404

Change History (1)

comment:1 Changed 4 years ago by nicolas.cellier.aka.nice@…

Description: modified (diff)
Note: See TracTickets for help on using tickets.