Fortran 90, changing 8 bit stream into a 10 bit one

[mediocre]

Hello,

I'm having a bit of trouble converting an 8 bit stream into a 10 bit one.

First i create intvalbin8, a Nx8 zero/one array where N is the byte length of the input data:
(6198 is the header offset factor)

j=1
do while (intvalout(i).gt.0)
 intvalbin8(j,i-6198)=modulo(intvalout(i),2)
 intvalout(i)=floor(real(intvalout(i))/2.)
 j=j+1
end do

Then i (hopefully) reshape it into a Nx10 one (intvalbin10) :

intvalbin10=reshape(intvalbin8,shape(intvalbin10))

Transforming it into 10 bit integers with:

intvaldec10=0
do i=1,size(intvaldec10)
 do j=1,10
  intvaldec10(i)=intvaldec10(i)+intvalbin10(j,i)*2**(j-1)
 end do
end do

In my case this is a column of values, corresponding to a 3712 x 464 array.
The problem is that it somehow messes up the ordering od the indices.

When printing out only every fourth line, the image looks "fine".
(this is the most decent looking out of the four subsets)

)

This is the full, jumbled array

I have no clue whatsoever what is wrong here. I've tried transposing,flipping... the 8bit one while reshaping it, but to no avail.

Any help is much appreciated.

Kind regards,

Marko

 

[gsal]

I don't understand what you are trying to do.

Can you show me an original "5x8" matrix, how it is supposed to look like and then show me how it is that you expect the 4x10 matrix to look like?

I am choosing a small matrix to so that you can put it all together by hand, of course.

Oops...I just realized that your Nx8 matrix is 2D and your 10-bit is a column vector...so, show me an original 5x8 and the corresponding expected vector...

 

[mediocre]

When reading data from a binary file, i only know how to read it in 1 byte chunks.

The file's header is written in this format (first 6198 entries), but the rest of is in 10 bit format.

That is why i opted for transforming the 8 bit stream into a 10 bit one, and then calulating the int values of the 10 bit one.

So, at first i have a an array Nx8 binary values:

$$\begin{bmatrix} b_{11} & b_{12} & ... \\ b_{21} & ... & ... \\ ... & ... & ... \\ b_{81} & ... & b_{8N} \end{bmatrix}$$

And then a corresponding 10 bit one should be (hopefully) :

$$\begin{bmatrix} b_{11} & b_{12} & ... \\ b_{21} & ... & ... \\ ... & ... & ... \\ b_{12} & ... & ... \\ b_{22} & ... & b_{8N_{10}} \end{bmatrix}, N_{10} = N * 8 /10$$

Each column would then represent a 10 bit number from the original bit stream.

A more elegant solution would be just to read the file in 10 bit chunks, but i am not sure how to do it. Is there a way of defining integer variables which are 10 bits in size?

Hopefully I've made it a bit more clear.

 

[mediocre]

Maybe a bit more clear would be to write it like this.

Instead of reading it like:

00001111 00001111 0000111...

I would like to read it like this:

0000111100 0011110000 111...

 

[rcgldr]

Assuming you have enough ram, it might be easier to read in the entire file into memory, then parse the binary image into 10 bit values, using a bit and byte offsets, and using 2 or 3 bytes in a row to pick up 10 bit values.

The bit offset would go from bit 0 to bit 7, and the byte offset would increase each time the bit offset increases over 8, where you subtract 8 from the bit offset and add 1 to the byte offset. After picking up a 10 bit value, you add 2 to the bit offset, and 1 to the byte offset. If the bit offset become 8 or greater, subtract 8 from the bit offset and add 1 to the byte offset.

The 3 byte case occurs when the bit offset is 7, but you could always pick up 3 bytes and end up effectively shifting the 3rd byte bits off the end most of the time. The code would look something like this

assuming you have a byte array that contains the binary image of the data as unsigned characters, and that integers are 32 or 64 bits:

character*1 mem[...]
integer mem0, mem1, mem2
integer xbits, x10bitvalue

mem0 = mem[byte_offset]
mem1 = mem[byte_offset+1]
mem2 = mem[byte_offset+2]
xbits = (mem0*65536) + (mem1*256) + mem2
xbits = xbits / (2**(14 - bit_offset))
x10bitvalue = modulo(xbits, 2**10)

byte_offset = byte_offset + 1
bit_offset = bit_offset + 2
if(bit_offset .GE. 8)then
bit_offset = bit_offset - 8
byte_offset = byte_offset+1
endif

The logic for writing 10 bit values would be somewhat the reverse of this process, except you would zero memory, the "add" the 10 bit values to memory:

mem0 = mem[byte_offset]
mem1 = mem[byte_offset+1]
mem2 = mem[byte_offset+2]

xbits = (mem0*65536) + (mem1*256) + mem2
xbits = xbits + (x10bitvalue*2**(14-bit_offset))
mem0 = xbits/65536
mem1 = modulo((xbits/256), 256)
mem2 = modulo(xbits,256)
mem[byte_offset] = mem0
mem[byte_offset+1] = mem1
mem[byte_offset+2] = mem2

Note if bit offset can never start on an odd bit boundary, then you only need to deal with 2 bytes at a time, readling would look like this:

mem0 = mem[byte_offset]
mem1 = mem[byte_offset+1]
xbits = (mem0*256) + mem1
xbits = xbits / (2**(6 - bit_offset))
x10bitvalue = modulo(xbits, 2**10)

 

[mediocre]

Happy new year, sorry for the late reply.

Thank you very much for the detailed explanation.

How would i go about reading the file as a bit sequence?

 

[rcgldr]

How would i go about reading the file as a bit sequence?

Just read it as a binary byte sequence (an array of bytes equal to the size of the file, which may include some padding at the end if the 10 bit values don't end up at an 8 bit (byte) boundary.

 

[mediocre]

Thank you very much for your help so far.

I'll try that out and see if it works.

Kind regards,

Marko