Mailing List Archive: Python: Python

numpy.frombuffer != unpack() ??

Index | Next | Previous | View Threaded

marlin_rowley at hotmail

May 16, 2008, 3:15 PM

Post #1 of 12 (375 views)
Permalink

numpy.frombuffer != unpack() ??

All:

I'm getting different floating point values when I use numpy vs. unpack().

frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
buffer = unpack("!f", byte)

frgba[0] != buffer[0]

why? This is forcing me use the unpack() function since it's giving me the correct values. What am I doing wrong?

-M

_________________________________________________________________
Change the world with e-mail. Join the i’m Initiative from Microsoft.
http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_ChangeWorld

robert.kern at gmail

May 16, 2008, 3:25 PM

Post #2 of 12 (367 views)
Permalink

Re: numpy.frombuffer != unpack() ?? [In reply to]

Marlin Rowley wrote:
> All:
>
> I'm getting different floating point values when I use numpy vs. unpack().
>
> frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> buffer = unpack("!f", byte)
>
> frgba[0] != buffer[0]
>
> why? This is forcing me use the unpack() function since it's giving me
> the correct values. What am I doing wrong?

Endianness, perhaps? '!' specifies big-endian data (an alias for '>'). Most
likely, you are on a little-endian platform. All of the dtypes in numpy default
to the native-endianness unless specified. If you want to read big-endian data
using numpy, do this:

frgba = numpy.frombuffer(<string of bytes>, dtype='>f')

If you have any more problems with numpy, please join us on the numpy mailing
list. When reporting problems, please try to provide a small but complete
snippet of self-contained code, the output that you got, and explain the output
that you expected to get. Thank you.

http://www.scipy.org/Mailing_Lists

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

marlin_rowley at hotmail

May 16, 2008, 3:31 PM

Post #3 of 12 (362 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

Thank you! That solved it!

-M

> To: python-list [at] python> From: robert.kern [at] gmail> Subject: Re: numpy.frombuffer != unpack() ??> Date: Fri, 16 May 2008 17:25:00 -0500> > Marlin Rowley wrote:> > All:> > > > I'm getting different floating point values when I use numpy vs. unpack().> > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)> > buffer = unpack("!f", byte)> > > > frgba[0] != buffer[0]> > > > why? This is forcing me use the unpack() function since it's giving me > > the correct values. What am I doing wrong?> > Endianness, perhaps? '!' specifies big-endian data (an alias for '>'). Most > likely, you are on a little-endian platform. All of the dtypes in numpy default > to the native-endianness unless specified. If you want to read big-endian data > using numpy, do this:> > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')> > If you have any more problems with numpy, please join us on the numpy mailing > list. When reporting problems, please try to provide a small but complete > snippet of self-contained code, the output that you got, and explain the output > that you expected to get. Thank you.> > http://www.scipy.org/Mailing_Lists> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless enigma> that is made terrible by our own mad attempt to interpret it as though it had> an underlying truth."> -- Umberto Eco> > --> http://mail.python.org/mailman/listinfo/python-list
_________________________________________________________________
E-mail for the greater good. Join the i’m Initiative from Microsoft.
http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_ GreaterGood

marlin_rowley at hotmail

May 16, 2008, 4:24 PM

Post #4 of 12 (363 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

All: Say I have an array: a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa']) How do I make it so that I now have: starting with first element (a[0])new_arr[0] = 'r'new_arr[1] = 'g'new_arr[2] = 'b'new_arr[3] = 'a'new_arr[4] = 'r'..... continuing "through" a[1] with the same new_arrnew_arr[N] = 'r'new_arr[N+1] = 'g'.... -M

From: marlin_rowley [at] hotmail: robert.kern [at] gmail; python-list [at] python: RE: numpy.frombuffer != unpack() ??Date: Fri, 16 May 2008 17:31:30 -0500

Thank you! That solved it! -M

> To: python-list [at] python> From: robert.kern [at] gmail> Subject: Re: numpy.frombuffer != unpack() ??> Date: Fri, 16 May 2008 17:25:00 -0500> > Marlin Rowley wrote:> > All:> > > > I'm getting different floating point values when I use numpy vs. unpack().> > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)> > buffer = unpack("!f", byte)> > > > frgba[0] != buffer[0]> > > > why? This is forcing me use the unpack() function since it's giving me > > the correct values. What am I doing wrong?> > Endianness, perhaps? '!' specifies big-endian data (an alias for '>'). Most > likely, you are on a little-endian platform. All of the dtypes in numpy default > to the native-endianness unless specified. If you want to read big-endian data > using numpy, do this:> > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')> > If you have any more problems with numpy, please join us on the numpy mailing > list. When reporting problems, please try to provide a small but complete > snippet of self-contained code, the output that you got, and explain the output > that you expected to get. Thank you.> > http://www.scipy.org/Mailing_Lists> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless enigma> that is made terrible by our own mad attempt to interpret it as though it had> an underlying truth."> -- Umberto Eco> > --> http://mail.python.org/mailman/listinfo/python-list

E-mail for the greater good. Join the i’m Initiative from Microsoft.
_________________________________________________________________
E-mail for the greater good. Join the i’m Initiative from Microsoft.
http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_ GreaterGood

robert.kern at gmail

May 16, 2008, 4:50 PM

Post #5 of 12 (369 views)
Permalink

Re: numpy.frombuffer != unpack() ?? [In reply to]

Marlin Rowley wrote:
> All:
>
> Say I have an array:
>
> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])

I'm sorry, you are going to have to be less elliptical. Exactly what do you have
there? It's certainly not an array, but a tuple of single-element lists each
containing a single string which can be interpreted as a group of 4 IEEE-754
single-precision floats. Is this correct? Do you really have single-element
lists? Some more context would be nice (can the tuple be longer? can the lists
be longer? can the strings be longer? etc.).

But basically, I would concatenate all of the strings together in the correct
order and use numpy.fromstring() on the resulting string.

And please, just respond to the list. I read this through GMane, and I don't
like getting duplicates in my in-box.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

gherron at islandtraining

May 16, 2008, 5:08 PM

Post #6 of 12 (357 views)
Permalink

Re: numpy.frombuffer != unpack() ?? [In reply to]

Marlin Rowley wrote:
> All:
>
> Say I have an array:
>
> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
>
> How do I make it so that I now have:
>
> starting with first element (a[0])
> new_arr[0] = 'r'
> new_arr[1] = 'g'
> new_arr[2] = 'b'
> new_arr[3] = 'a'
> new_arr[4] = 'r'
> .....
>
> continuing "through" a[1] with the same new_arr
> new_arr[N] = 'r'
> new_arr[N+1] = 'g'
> ....
>
> -M

Numpy can do this for you. First, do you really mean the array to
contain lists of one string each? If so:

>>> import numpy
>>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
>>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
kludge here
>>> b
array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b', 'a',
'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
'b', 'b', 'a', 'a', 'a', 'a'],
dtype='|S1')
>>> b.shape=(2,4,4)
>>> b
array([[['r', 'r', 'r', 'r'],
['g', 'g', 'g', 'g'],
['b', 'b', 'b', 'b'],
['a', 'a', 'a', 'a']],

[['r', 'r', 'r', 'r'],
['g', 'g', 'g', 'g'],
['b', 'b', 'b', 'b'],
['a', 'a', 'a', 'a']]],
dtype='|S1')
>>> c = b.transpose((2,0,1))
>>> c
array([[['r', 'g', 'b', 'a'],
['r', 'g', 'b', 'a']],

[['r', 'g', 'b', 'a'],
['r', 'g', 'b', 'a']],

[['r', 'g', 'b', 'a'],
['r', 'g', 'b', 'a']],

[['r', 'g', 'b', 'a'],
['r', 'g', 'b', 'a']]],
dtype='|S1')
>>> d=c.copy() # To make it contiguous
>>> d.shape = (32,)
>>> d
array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r',
'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
'b', 'a', 'r', 'g', 'b', 'a'],
dtype='|S1')

Done. Cool no?

Gary Herron

>
>
>
>
> ------------------------------------------------------------------------
> From: marlin_rowley [at] hotmail
> To: robert.kern [at] gmail; python-list [at] python
> Subject: RE: numpy.frombuffer != unpack() ??
> Date: Fri, 16 May 2008 17:31:30 -0500
>
> Thank you! That solved it!
>
> -M
>
>
> ------------------------------------------------------------------------
>
> > To: python-list [at] python
> > From: robert.kern [at] gmail
> > Subject: Re: numpy.frombuffer != unpack() ??
> > Date: Fri, 16 May 2008 17:25:00 -0500
> >
> > Marlin Rowley wrote:
> > > All:
> > >
> > > I'm getting different floating point values when I use numpy
> vs. unpack().
> > >
> > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> > > buffer = unpack("!f", byte)
> > >
> > > frgba[0] != buffer[0]
> > >
> > > why? This is forcing me use the unpack() function since it's
> giving me
> > > the correct values. What am I doing wrong?
> >
> > Endianness, perhaps? '!' specifies big-endian data (an alias for
> '>'). Most
> > likely, you are on a little-endian platform. All of the dtypes
> in numpy default
> > to the native-endianness unless specified. If you want to read
> big-endian data
> > using numpy, do this:
> >
> > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
> >
> > If you have any more problems with numpy, please join us on the
> numpy mailing
> > list. When reporting problems, please try to provide a small but
> complete
> > snippet of self-contained code, the output that you got, and
> explain the output
> > that you expected to get. Thank you.
> >
> > http://www.scipy.org/Mailing_Lists
> >
> > --
> > Robert Kern
> >
> > "I have come to believe that the whole world is an enigma, a
> harmless enigma
> > that is made terrible by our own mad attempt to interpret it as
> though it had
> > an underlying truth."
> > -- Umberto Eco
> >
> > --
> > http://mail.python.org/mailman/listinfo/python-list
>
>
> ------------------------------------------------------------------------
> E-mail for the greater good. Join the i’m Initiative from
> Microsoft.
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
>
>
>
> ------------------------------------------------------------------------
> E-mail for the greater good. Join the i’m Initiative from Microsoft.
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
>
> ------------------------------------------------------------------------
>
> --
> http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

marlin_rowley at hotmail

May 16, 2008, 5:22 PM

Post #7 of 12 (363 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

Very cool.

> > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
a represents a tile with height of 2 and width of 4 with 4 bits/pixel for each color.

> >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
this seperates the stream into individual values - Check

> >>> b.shape=(2,4,4)
This reshapes the array so that b.shape=(height,width,#bits/pixel) - Check

>>> c = b.transpose((2,0,1))

What does the (2,0,1) represent in terms of width and height and number of bytes?

> Date: Fri, 16 May 2008 17:08:20 -0700> From: gherron [at] islandtraining> To: marlin_rowley [at] hotmail; python-list [at] python> Subject: Re: numpy.frombuffer != unpack() ??> > Marlin Rowley wrote:> > All:> > > > Say I have an array:> > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > > How do I make it so that I now have:> > > > starting with first element (a[0])> > new_arr[0] = 'r'> > new_arr[1] = 'g'> > new_arr[2] = 'b'> > new_arr[3] = 'a'> > new_arr[4] = 'r'> > .....> > > > continuing "through" a[1] with the same new_arr> > new_arr[N] = 'r'> > new_arr[N+1] = 'g'> > ....> > > > -M> > Numpy can do this for you. First, do you really mean the array to > contain lists of one string each? If so:> > >>> import numpy> >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a > kludge here> >>> b> array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b', 'a',> 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',> 'b', 'b', 'a', 'a', 'a', 'a'],> dtype='|S1')> >>> b.shape=(2,4,4)> >>> b> array([[['r', 'r', 'r', 'r'],> ['g', 'g', 'g', 'g'],> ['b', 'b', 'b', 'b'],> ['a', 'a', 'a', 'a']],> > [['r', 'r', 'r', 'r'],> ['g', 'g', 'g', 'g'],> ['b', 'b', 'b', 'b'],> ['a', 'a', 'a', 'a']]],> dtype='|S1')> >>> c = b.transpose((2,0,1))> >>> c> array([[['r', 'g', 'b', 'a'],> ['r', 'g', 'b', 'a']],> > [['r', 'g', 'b', 'a'],> ['r', 'g', 'b', 'a']],> > [['r', 'g', 'b', 'a'],> ['r', 'g', 'b', 'a']],> > [['r', 'g', 'b', 'a'],> ['r', 'g', 'b', 'a']]],> dtype='|S1')> >>> d=c.copy() # To make it contiguous> >>> d.shape = (32,)> >>> d> array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r',> 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',> 'b', 'a', 'r', 'g', 'b', 'a'],> dtype='|S1')> > Done. Cool no?> > Gary Herron> > >> >> >> >> > ------------------------------------------------------------------------> > From: marlin_rowley [at] hotmail> > To: robert.kern [at] gmail; python-list [at] python> > Subject: RE: numpy.frombuffer != unpack() ??> > Date: Fri, 16 May 2008 17:31:30 -0500> >> > Thank you! That solved it!> > > > -M> >> >> > ------------------------------------------------------------------------> >> > > To: python-list [at] python> > > From: robert.kern [at] gmail> > > Subject: Re: numpy.frombuffer != unpack() ??> > > Date: Fri, 16 May 2008 17:25:00 -0500> > >> > > Marlin Rowley wrote:> > > > All:> > > >> > > > I'm getting different floating point values when I use numpy> > vs. unpack().> > > >> > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)> > > > buffer = unpack("!f", byte)> > > >> > > > frgba[0] != buffer[0]> > > >> > > > why? This is forcing me use the unpack() function since it's> > giving me> > > > the correct values. What am I doing wrong?> > >> > > Endianness, perhaps? '!' specifies big-endian data (an alias for> > '>'). Most> > > likely, you are on a little-endian platform. All of the dtypes> > in numpy default> > > to the native-endianness unless specified. If you want to read> > big-endian data> > > using numpy, do this:> > >> > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')> > >> > > If you have any more problems with numpy, please join us on the> > numpy mailing> > > list. When reporting problems, please try to provide a small but> > complete> > > snippet of self-contained code, the output that you got, and> > explain the output> > > that you expected to get. Thank you.> > >> > > http://www.scipy.org/Mailing_Lists> > >> > > --> > > Robert Kern> > >> > > "I have come to believe that the whole world is an enigma, a> > harmless enigma> > > that is made terrible by our own mad attempt to interpret it as> > though it had> > > an underlying truth."> > > -- Umberto Eco> > >> > > --> > > http://mail.python.org/mailman/listinfo/python-list> >> >> > ------------------------------------------------------------------------> > E-mail for the greater good. Join the i’m Initiative from> > Microsoft.> > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>> >> >> >> > ------------------------------------------------------------------------> > E-mail for the greater good. Join the i’m Initiative from Microsoft. > > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood> > >> > ------------------------------------------------------------------------> >> > --> > http://mail.python.org/mailman/listinfo/python-list>
_________________________________________________________________
Keep your kids safer online with Windows Live Family Safety.
http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008

marlin_rowley at hotmail

May 16, 2008, 5:36 PM

Post #8 of 12 (367 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

Ok. Here's the deal..

I've gathered a string of bytes that are turned into floats and stored in an array of floats.

The stream gives me color channels represented in bytes.

stream = 'rrrrrrrrggggggggggbbbbbbbbbaaaaaaaaarrrrrrrrrrrggggggggggbbbbbbbbbbaaaaaaaa......' etc..

the number of r's,g's,b's in one consecutive pattern represent the width of a bitmap. The number of times the pattern (rgba) is repeated gives the height of the bitmap. I must first convert these bytes into floats. The rgba represents floating point values for each of the pixel components. I've gotten that far.

Now I have an array of individual floats with that pattern. I need to:

1) pull out the alphas from this float list.
2) reshape this float list so that I have an array that seperates the repeated pattern into a height of this bitmap:

rrrrrrrrrrrgggggggggggbbbbbbbbb [0] - first pattern
...
rrrrrrrrrrrgggggggggggbbbbbbbbb [height-1] - last pattern

3) then I need to reverse this array so that height-1 is the first element in my new array:

rrrrrrrrrrrgggggggggggbbbbbbbbb [height-1]
...
rrrrrrrrrrrgggggggggggbbbbbbbbb [0]

since wx.Python reverses the y-axis in a bitmap

4) Then I need to make the pattern another pattern:
rrrrrrrrrrrgggggggggggbbbbbbbbb ---> needs to become ---> rgbrgbrgb.... starting at [height-1] (my first pattern in my new array).

5) Then I need to flatten the array to one continuous array of floats:

rgbrgbrgbrgb[from height-1]............rgbrgbrgb[height = 0]

this is my final array to be drawn on the screen!

> To: python-list [at] python> From: robert.kern [at] gmail> Subject: Re: numpy.frombuffer != unpack() ??> Date: Fri, 16 May 2008 18:50:38 -0500> > Marlin Rowley wrote:> > All:> > > > Say I have an array:> > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > I'm sorry, you are going to have to be less elliptical. Exactly what do you have > there? It's certainly not an array, but a tuple of single-element lists each > containing a single string which can be interpreted as a group of 4 IEEE-754 > single-precision floats. Is this correct? Do you really have single-element > lists? Some more context would be nice (can the tuple be longer? can the lists > be longer? can the strings be longer? etc.).> > But basically, I would concatenate all of the strings together in the correct > order and use numpy.fromstring() on the resulting string.> > And please, just respond to the list. I read this through GMane, and I don't > like getting duplicates in my in-box.> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless enigma> that is made terrible by our own mad attempt to interpret it as though it had> an underlying truth."> -- Umberto Eco> > --> http://mail.python.org/mailman/listinfo/python-list
_________________________________________________________________
Keep your kids safer online with Windows Live Family Safety.
http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008

gherron at islandtraining

May 17, 2008, 8:58 AM

Post #9 of 12 (342 views)
Permalink

Re: numpy.frombuffer != unpack() ?? [In reply to]

Marlin Rowley wrote:
>
> Very cool.
>
> > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> a represents a tile with height of 2 and width of 4 with 4 bits/pixel
> for each color.
>
> > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
> this seperates the stream into individual values - Check
>
> > >>> b.shape=(2,4,4)
>
> This reshapes the array so that b.shape=(height,width,#bits/pixel) - Check
>
> >>> c = b.transpose((2,0,1))
>
> What does the (2,0,1) represent in terms of width and height and
> number of bytes?

The (2,0,1) tells how to exchange the axes. For instance in a 2D array,
a normal transpose exchanges rows and columns. It will change a (a by
b) sized array into a (b by a) sized array. This would be equivalent to
the more saying interchange axes 0,1 to the new order of 1,0.

In numpy with higher dimension arrays, the default transpose just
exchanges the first two axes, and the full transpose allows you to
specify exactly the new ordering of the exes.

So transpose((2,0,1)) means take axes (0,1,2) to the new order
(2,1,0). In terms of sizes, an (a by b by c) sized array will end being
of size (c by a by b) in size.

In terms of implementation, there may not be *any* data re-arrangement
in a transpose. The only thing that needs changing is how the indices
are converted to an actual machine address of an indexed item. The
documentation notes this by saying transpose returns a "new view" of the
array. This explains why I copied the array before extracting bytes
out of it -- you really do need the elements in the new order for the
next operation.

Gary Herron

>
>
>
>
>
> ------------------------------------------------------------------------
>
> > Date: Fri, 16 May 2008 17:08:20 -0700
> > From: gherron [at] islandtraining
> > To: marlin_rowley [at] hotmail; python-list [at] python
> > Subject: Re: numpy.frombuffer != unpack() ??
> >
> > Marlin Rowley wrote:
> > > All:
> > >
> > > Say I have an array:
> > >
> > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > >
> > > How do I make it so that I now have:
> > >
> > > starting with first element (a[0])
> > > new_arr[0] = 'r'
> > > new_arr[1] = 'g'
> > > new_arr[2] = 'b'
> > > new_arr[3] = 'a'
> > > new_arr[4] = 'r'
> > > .....
> > >
> > > continuing "through" a[1] with the same new_arr
> > > new_arr[N] = 'r'
> > > new_arr[N+1] = 'g'
> > > ....
> > >
> > > -M
> >
> > Numpy can do this for you. First, do you really mean the array to
> > contain lists of one string each? If so:
> >
> > >>> import numpy
> > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
> > kludge here
> > >>> b
> > array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b', 'a',
> > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
> > 'b', 'b', 'a', 'a', 'a', 'a'],
> > dtype='|S1')
> > >>> b.shape=(2,4,4)
> > >>> b
> > array([[['r', 'r', 'r', 'r'],
> > ['g', 'g', 'g', 'g'],
> > ['b', 'b', 'b', 'b'],
> > ['a', 'a', 'a', 'a']],
> >
> > [['r', 'r', 'r', 'r'],
> > ['g', 'g', 'g', 'g'],
> > ['b', 'b', 'b', 'b'],
> > ['a', 'a', 'a', 'a']]],
> > dtype='|S1')
> > >>> c = b.transpose((2,0,1))
> > >>> c
> > array([[['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']]],
> > dtype='|S1')
> > >>> d=c.copy() # To make it contiguous
> > >>> d.shape = (32,)
> > >>> d
> > array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r',
> > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
> > 'b', 'a', 'r', 'g', 'b', 'a'],
> > dtype='|S1')
> >
> > Done. Cool no?
> >
> > Gary Herron
> >
> > >
> > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > From: marlin_rowley [at] hotmail
> > > To: robert.kern [at] gmail; python-list [at] python
> > > Subject: RE: numpy.frombuffer != unpack() ??
> > > Date: Fri, 16 May 2008 17:31:30 -0500
> > >
> > > Thank you! That solved it!
> > >
> > > -M
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > >
> > > > To: python-list [at] python
> > > > From: robert.kern [at] gmail
> > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > > Date: Fri, 16 May 2008 17:25:00 -0500
> > > >
> > > > Marlin Rowley wrote:
> > > > > All:
> > > > >
> > > > > I'm getting different floating point values when I use numpy
> > > vs. unpack().
> > > > >
> > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> > > > > buffer = unpack("!f", byte)
> > > > >
> > > > > frgba[0] != buffer[0]
> > > > >
> > > > > why? This is forcing me use the unpack() function since it's
> > > giving me
> > > > > the correct values. What am I doing wrong?
> > > >
> > > > Endianness, perhaps? '!' specifies big-endian data (an alias for
> > > '>'). Most
> > > > likely, you are on a little-endian platform. All of the dtypes
> > > in numpy default
> > > > to the native-endianness unless specified. If you want to read
> > > big-endian data
> > > > using numpy, do this:
> > > >
> > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
> > > >
> > > > If you have any more problems with numpy, please join us on the
> > > numpy mailing
> > > > list. When reporting problems, please try to provide a small but
> > > complete
> > > > snippet of self-contained code, the output that you got, and
> > > explain the output
> > > > that you expected to get. Thank you.
> > > >
> > > > http://www.scipy.org/Mailing_Lists
> > > >
> > > > --
> > > > Robert Kern
> > > >
> > > > "I have come to believe that the whole world is an enigma, a
> > > harmless enigma
> > > > that is made terrible by our own mad attempt to interpret it as
> > > though it had
> > > > an underlying truth."
> > > > -- Umberto Eco
> > > >
> > > > --
> > > > http://mail.python.org/mailman/listinfo/python-list
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > E-mail for the greater good. Join the i’m Initiative from
> > > Microsoft.
> > >
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
> > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > E-mail for the greater good. Join the i’m Initiative from Microsoft.
> > >
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
>
> > >
> > >
> ------------------------------------------------------------------------
> > >
> > > --
> > > http://mail.python.org/mailman/listinfo/python-list
> >
>
>
> ------------------------------------------------------------------------
> Keep your kids safer online with Windows Live Family Safety. Help
> protect your kids.
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008>
>
> ------------------------------------------------------------------------
>
> --
> http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

marlin_rowley at hotmail

May 17, 2008, 2:45 PM

Post #10 of 12 (335 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

Actually in my traversal of the never-ending maze of understanding arrays in python, I stumbled that the data in the wrong sequence.

Let's get back to this transpose() deal, say we have these values as integers (representing rgba again): [[[0,1,2,3][4,5,6,7][8,9,10,11][12,13,14,15]] [[16,17,18,19][20,21,22,23][24,25,26,27][28,29,30,31]]] Now if I do this: transpose((2,0,1)), I get this: [[[0,4,8,12] [16,20,24,28]][[1,5,9,13] [17,21,25,29]][[2,6,10,14][18,22,26,30]][[3,7,11,15][19,23,27,31]]] This is NOT what I want. I want the new array to be: [0,4,8,12][1,5,9,13][2,6,10,14][3,7,11,15][16,20,24,28][17,21,25,29][18,22,26,30][19,23,27,31] How do I do this? -M

> Date: Sat, 17 May 2008 08:58:08 -0700> From: gherron [at] islandtraining> To: marlin_rowley [at] hotmail> CC: python-list [at] python> Subject: Re: numpy.frombuffer != unpack() ??> > Marlin Rowley wrote:> > > > Very cool.> > > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > a represents a tile with height of 2 and width of 4 with 4 bits/pixel > > for each color.> > > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')> > this seperates the stream into individual values - Check> > > > > >>> b.shape=(2,4,4)> >> > This reshapes the array so that b.shape=(height,width,#bits/pixel) - Check> > > > >>> c = b.transpose((2,0,1))> > > > What does the (2,0,1) represent in terms of width and height and > > number of bytes?> > The (2,0,1) tells how to exchange the axes. For instance in a 2D array, > a normal transpose exchanges rows and columns. It will change a (a by > b) sized array into a (b by a) sized array. This would be equivalent to > the more saying interchange axes 0,1 to the new order of 1,0. > > In numpy with higher dimension arrays, the default transpose just > exchanges the first two axes, and the full transpose allows you to > specify exactly the new ordering of the exes. > > So transpose((2,0,1)) means take axes (0,1,2) to the new order > (2,1,0). In terms of sizes, an (a by b by c) sized array will end being > of size (c by a by b) in size. > > In terms of implementation, there may not be *any* data re-arrangement > in a transpose. The only thing that needs changing is how the indices > are converted to an actual machine address of an indexed item. The > documentation notes this by saying transpose returns a "new view" of the > array. This explains why I copied the array before extracting bytes > out of it -- you really do need the elements in the new order for the > next operation.> > Gary Herron> > >> >> >> >> > > > ------------------------------------------------------------------------> >> > > Date: Fri, 16 May 2008 17:08:20 -0700> > > From: gherron [at] islandtraining> > > To: marlin_rowley [at] hotmail; python-list [at] python> > > Subject: Re: numpy.frombuffer != unpack() ??> > >> > > Marlin Rowley wrote:> > > > All:> > > >> > > > Say I have an array:> > > >> > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > >> > > > How do I make it so that I now have:> > > >> > > > starting with first element (a[0])> > > > new_arr[0] = 'r'> > > > new_arr[1] = 'g'> > > > new_arr[2] = 'b'> > > > new_arr[3] = 'a'> > > > new_arr[4] = 'r'> > > > .....> > > >> > > > continuing "through" a[1] with the same new_arr> > > > new_arr[N] = 'r'> > > > new_arr[N+1] = 'g'> > > > ....> > > >> > > > -M> > >> > > Numpy can do this for you. First, do you really mean the array to> > > contain lists of one string each? If so:> > >> > > >>> import numpy> > > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a> > > kludge here> > > >>> b> > > array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b', 'a',> > > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',> > > 'b', 'b', 'a', 'a', 'a', 'a'],> > > dtype='|S1')> > > >>> b.shape=(2,4,4)> > > >>> b> > > array([[['r', 'r', 'r', 'r'],> > > ['g', 'g', 'g', 'g'],> > > ['b', 'b', 'b', 'b'],> > > ['a', 'a', 'a', 'a']],> > >> > > [['r', 'r', 'r', 'r'],> > > ['g', 'g', 'g', 'g'],> > > ['b', 'b', 'b', 'b'],> > > ['a', 'a', 'a', 'a']]],> > > dtype='|S1')> > > >>> c = b.transpose((2,0,1))> > > >>> c> > > array([[['r', 'g', 'b', 'a'],> > > ['r', 'g', 'b', 'a']],> > >> > > [['r', 'g', 'b', 'a'],> > > ['r', 'g', 'b', 'a']],> > >> > > [['r', 'g', 'b', 'a'],> > > ['r', 'g', 'b', 'a']],> > >> > > [['r', 'g', 'b', 'a'],> > > ['r', 'g', 'b', 'a']]],> > > dtype='|S1')> > > >>> d=c.copy() # To make it contiguous> > > >>> d.shape = (32,)> > > >>> d> > > array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r',> > > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',> > > 'b', 'a', 'r', 'g', 'b', 'a'],> > > dtype='|S1')> > >> > > Done. Cool no?> > >> > > Gary Herron> > >> > > >> > > >> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > > From: marlin_rowley [at] hotmail> > > > To: robert.kern [at] gmail; python-list [at] python> > > > Subject: RE: numpy.frombuffer != unpack() ??> > > > Date: Fri, 16 May 2008 17:31:30 -0500> > > >> > > > Thank you! That solved it!> > > >> > > > -M> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > >> > > > > To: python-list [at] python> > > > > From: robert.kern [at] gmail> > > > > Subject: Re: numpy.frombuffer != unpack() ??> > > > > Date: Fri, 16 May 2008 17:25:00 -0500> > > > >> > > > > Marlin Rowley wrote:> > > > > > All:> > > > > >> > > > > > I'm getting different floating point values when I use numpy> > > > vs. unpack().> > > > > >> > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)> > > > > > buffer = unpack("!f", byte)> > > > > >> > > > > > frgba[0] != buffer[0]> > > > > >> > > > > > why? This is forcing me use the unpack() function since it's> > > > giving me> > > > > > the correct values. What am I doing wrong?> > > > >> > > > > Endianness, perhaps? '!' specifies big-endian data (an alias for> > > > '>'). Most> > > > > likely, you are on a little-endian platform. All of the dtypes> > > > in numpy default> > > > > to the native-endianness unless specified. If you want to read> > > > big-endian data> > > > > using numpy, do this:> > > > >> > > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')> > > > >> > > > > If you have any more problems with numpy, please join us on the> > > > numpy mailing> > > > > list. When reporting problems, please try to provide a small but> > > > complete> > > > > snippet of self-contained code, the output that you got, and> > > > explain the output> > > > > that you expected to get. Thank you.> > > > >> > > > > http://www.scipy.org/Mailing_Lists> > > > >> > > > > --> > > > > Robert Kern> > > > >> > > > > "I have come to believe that the whole world is an enigma, a> > > > harmless enigma> > > > > that is made terrible by our own mad attempt to interpret it as> > > > though it had> > > > > an underlying truth."> > > > > -- Umberto Eco> > > > >> > > > > --> > > > > http://mail.python.org/mailman/listinfo/python-list> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > > E-mail for the greater good. Join the i’m Initiative from> > > > Microsoft.> > > > > > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>> > > >> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > > E-mail for the greater good. Join the i’m Initiative from Microsoft.> > > > > > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood> > >> > > >> > > > > > ------------------------------------------------------------------------> > > >> > > > --> > > > http://mail.python.org/mailman/listinfo/python-list> > >> >> >> > ------------------------------------------------------------------------> > Keep your kids safer online with Windows Live Family Safety. Help > > protect your kids. > > <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> > >> > ------------------------------------------------------------------------> >> > --> > http://mail.python.org/mailman/listinfo/python-list>
_________________________________________________________________
Keep your kids safer online with Windows Live Family Safety.
http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008

gherron at islandtraining

May 17, 2008, 5:18 PM

Post #11 of 12 (332 views)
Permalink

Re: numpy.frombuffer != unpack() ?? [In reply to]

Marlin Rowley wrote:
> Actually in my traversal of the never-ending maze of understanding
> arrays in python, I stumbled that the data in the wrong sequence.
>
> Let's get back to this transpose() deal, say we have these values as
> integers (representing rgba again):
>
> [[[0,1,2,3]
> [4,5,6,7]
> [8,9,10,11]
> [12,13,14,15]]
>
> [[16,17,18,19]
> [20,21,22,23]
> [24,25,26,27]
> [28,29,30,31]]]
>
> Now if I do this: transpose((2,0,1)), I get this:
>
> [[[0,4,8,12] [16,20,24,28]]
> [[1,5,9,13] [17,21,25,29]]
> [[2,6,10,14][18,22,26,30]]
> [[3,7,11,15][19,23,27,31]]]
>
> This is NOT what I want. I want the new array to be:
>
> [0,4,8,12][1,5,9,13]
> [2,6,10,14][3,7,11,15]
> [16,20,24,28][17,21,25,29]
> [18,22,26,30][19,23,27,31]
>
> How do I do this?

That's s little ambiguous, but one of the following two
transpose-reshape-print's might be what you want.

Gary Herron

import numpy

a = numpy.array([[[0,1,2,3],
[4,5,6,7],
[8,9,10,11],
[12,13,14,15]],

[[16,17,18,19],
[20,21,22,23],
[24,25,26,27],
[28,29,30,31]]])

print numpy.reshape(a.transpose(0,2,1), (8,4))
print numpy.reshape(a.transpose(0,2,1), (4,2,4))

output is:

[[ 0 4 8 12]
[ 1 5 9 13]
[ 2 6 10 14]
[ 3 7 11 15]
[16 20 24 28]
[17 21 25 29]
[18 22 26 30]
[19 23 27 31]]

and

[[[ 0 4 8 12]
[ 1 5 9 13]]

[[ 2 6 10 14]
[ 3 7 11 15]]

[[16 20 24 28]
[17 21 25 29]]

[[18 22 26 30]
[19 23 27 31]]]

>
> -M
>
>
>
>
>
> ------------------------------------------------------------------------
>
> > Date: Sat, 17 May 2008 08:58:08 -0700
> > From: gherron [at] islandtraining
> > To: marlin_rowley [at] hotmail
> > CC: python-list [at] python
> > Subject: Re: numpy.frombuffer != unpack() ??
> >
> > Marlin Rowley wrote:
> > >
> > > Very cool.
> > >
> > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > a represents a tile with height of 2 and width of 4 with 4 bits/pixel
> > > for each color.
> > >
> > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
> > > this seperates the stream into individual values - Check
> > >
> > > > >>> b.shape=(2,4,4)
> > >
> > > This reshapes the array so that b.shape=(height,width,#bits/pixel)
> - Check
> > >
> > > >>> c = b.transpose((2,0,1))
> > >
> > > What does the (2,0,1) represent in terms of width and height and
> > > number of bytes?
> >
> > The (2,0,1) tells how to exchange the axes. For instance in a 2D array,
> > a normal transpose exchanges rows and columns. It will change a (a by
> > b) sized array into a (b by a) sized array. This would be equivalent to
> > the more saying interchange axes 0,1 to the new order of 1,0.
> >
> > In numpy with higher dimension arrays, the default transpose just
> > exchanges the first two axes, and the full transpose allows you to
> > specify exactly the new ordering of the exes.
> >
> > So transpose((2,0,1)) means take axes (0,1,2) to the new order
> > (2,1,0). In terms of sizes, an (a by b by c) sized array will end being
> > of size (c by a by b) in size.
> >
> > In terms of implementation, there may not be *any* data re-arrangement
> > in a transpose. The only thing that needs changing is how the indices
> > are converted to an actual machine address of an indexed item. The
> > documentation notes this by saying transpose returns a "new view" of
> the
> > array. This explains why I copied the array before extracting bytes
> > out of it -- you really do need the elements in the new order for the
> > next operation.
> >
> > Gary Herron
> >
> > >
> > >
> > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > >
> > > > Date: Fri, 16 May 2008 17:08:20 -0700
> > > > From: gherron [at] islandtraining
> > > > To: marlin_rowley [at] hotmail; python-list [at] python
> > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > >
> > > > Marlin Rowley wrote:
> > > > > All:
> > > > >
> > > > > Say I have an array:
> > > > >
> > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > > >
> > > > > How do I make it so that I now have:
> > > > >
> > > > > starting with first element (a[0])
> > > > > new_arr[0] = 'r'
> > > > > new_arr[1] = 'g'
> > > > > new_arr[2] = 'b'
> > > > > new_arr[3] = 'a'
> > > > > new_arr[4] = 'r'
> > > > > .....
> > > > >
> > > > > continuing "through" a[1] with the same new_arr
> > > > > new_arr[N] = 'r'
> > > > > new_arr[N+1] = 'g'
> > > > > ....
> > > > >
> > > > > -M
> > > >
> > > > Numpy can do this for you. First, do you really mean the array to
> > > > contain lists of one string each? If so:
> > > >
> > > > >>> import numpy
> > > > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
> > > > kludge here
> > > > >>> b
> > > > array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b',
> 'b', 'a',
> > > > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
> > > > 'b', 'b', 'a', 'a', 'a', 'a'],
> > > > dtype='|S1')
> > > > >>> b.shape=(2,4,4)
> > > > >>> b
> > > > array([[['r', 'r', 'r', 'r'],
> > > > ['g', 'g', 'g', 'g'],
> > > > ['b', 'b', 'b', 'b'],
> > > > ['a', 'a', 'a', 'a']],
> > > >
> > > > [['r', 'r', 'r', 'r'],
> > > > ['g', 'g', 'g', 'g'],
> > > > ['b', 'b', 'b', 'b'],
> > > > ['a', 'a', 'a', 'a']]],
> > > > dtype='|S1')
> > > > >>> c = b.transpose((2,0,1))
> > > > >>> c
> > > > array([[['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']]],
> > > > dtype='|S1')
> > > > >>> d=c.copy() # To make it contiguous
> > > > >>> d.shape = (32,)
> > > > >>> d
> > > > array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b',
> 'a', 'r',
> > > > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
> > > > 'b', 'a', 'r', 'g', 'b', 'a'],
> > > > dtype='|S1')
> > > >
> > > > Done. Cool no?
> > > >
> > > > Gary Herron
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > > From: marlin_rowley [at] hotmail
> > > > > To: robert.kern [at] gmail; python-list [at] python
> > > > > Subject: RE: numpy.frombuffer != unpack() ??
> > > > > Date: Fri, 16 May 2008 17:31:30 -0500
> > > > >
> > > > > Thank you! That solved it!
> > > > >
> > > > > -M
> > > > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > > To: python-list [at] python
> > > > > > From: robert.kern [at] gmail
> > > > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > > > > Date: Fri, 16 May 2008 17:25:00 -0500
> > > > > >
> > > > > > Marlin Rowley wrote:
> > > > > > > All:
> > > > > > >
> > > > > > > I'm getting different floating point values when I use numpy
> > > > > vs. unpack().
> > > > > > >
> > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> > > > > > > buffer = unpack("!f", byte)
> > > > > > >
> > > > > > > frgba[0] != buffer[0]
> > > > > > >
> > > > > > > why? This is forcing me use the unpack() function since it's
> > > > > giving me
> > > > > > > the correct values. What am I doing wrong?
> > > > > >
> > > > > > Endianness, perhaps? '!' specifies big-endian data (an alias for
> > > > > '>'). Most
> > > > > > likely, you are on a little-endian platform. All of the dtypes
> > > > > in numpy default
> > > > > > to the native-endianness unless specified. If you want to read
> > > > > big-endian data
> > > > > > using numpy, do this:
> > > > > >
> > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
> > > > > >
> > > > > > If you have any more problems with numpy, please join us on the
> > > > > numpy mailing
> > > > > > list. When reporting problems, please try to provide a small but
> > > > > complete
> > > > > > snippet of self-contained code, the output that you got, and
> > > > > explain the output
> > > > > > that you expected to get. Thank you.
> > > > > >
> > > > > > http://www.scipy.org/Mailing_Lists
> > > > > >
> > > > > > --
> > > > > > Robert Kern
> > > > > >
> > > > > > "I have come to believe that the whole world is an enigma, a
> > > > > harmless enigma
> > > > > > that is made terrible by our own mad attempt to interpret it as
> > > > > though it had
> > > > > > an underlying truth."
> > > > > > -- Umberto Eco
> > > > > >
> > > > > > --
> > > > > > http://mail.python.org/mailman/listinfo/python-list
> > > > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > > E-mail for the greater good. Join the i’m Initiative from
> > > > > Microsoft.
> > > > >
> > >
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > > E-mail for the greater good. Join the i’m Initiative from
> Microsoft.
> > > > >
> > >
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
>
> > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > --
> > > > > http://mail.python.org/mailman/listinfo/python-list
> > > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > Keep your kids safer online with Windows Live Family Safety. Help
> > > protect your kids.
> > >
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008>
>
> > >
> > >
> ------------------------------------------------------------------------
> > >
> > > --
> > > http://mail.python.org/mailman/listinfo/python-list
> >
>
>
> ------------------------------------------------------------------------
> Keep your kids safer online with Windows Live Family Safety. Help
> protect your kids.
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008>
>
> ------------------------------------------------------------------------
>
> --
> http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

marlin_rowley at hotmail

May 17, 2008, 7:14 PM

Post #12 of 12 (335 views)
Permalink

RE: numpy.frombuffer != unpack() ?? [In reply to]

Gary,

That works. And it's better than my solution:

#a = numpy.rollaxis(a,2,0)
#a = numpy.rollaxis(a,0,1)
#a = numpy.rollaxis(a,1,0)

Thanks!

-M

> Date: Sat, 17 May 2008 17:18:39 -0700> From: gherron [at] islandtraining> To: marlin_rowley [at] hotmail> CC: python-list [at] python> Subject: Re: numpy.frombuffer != unpack() ??> > Marlin Rowley wrote:> > Actually in my traversal of the never-ending maze of understanding > > arrays in python, I stumbled that the data in the wrong sequence.> >> > Let's get back to this transpose() deal, say we have these values as > > integers (representing rgba again):> >> > [[[0,1,2,3]> > [4,5,6,7]> > [8,9,10,11]> > [12,13,14,15]]> >> > [[16,17,18,19]> > [20,21,22,23]> > [24,25,26,27]> > [28,29,30,31]]]> >> > Now if I do this: transpose((2,0,1)), I get this:> >> > [[[0,4,8,12] [16,20,24,28]]> > [[1,5,9,13] [17,21,25,29]]> > [[2,6,10,14][18,22,26,30]]> > [[3,7,11,15][19,23,27,31]]]> >> > This is NOT what I want. I want the new array to be:> >> > [0,4,8,12][1,5,9,13]> > [2,6,10,14][3,7,11,15]> > [16,20,24,28][17,21,25,29]> > [18,22,26,30][19,23,27,31]> >> > How do I do this?> > That's s little ambiguous, but one of the following two > transpose-reshape-print's might be what you want.> > Gary Herron> > > import numpy> > a = numpy.array([[[0,1,2,3],> [4,5,6,7],> [8,9,10,11],> [12,13,14,15]],> > [[16,17,18,19],> [20,21,22,23],> [24,25,26,27],> [28,29,30,31]]])> > > print numpy.reshape(a.transpose(0,2,1), (8,4))> print numpy.reshape(a.transpose(0,2,1), (4,2,4))> > output is:> > [[ 0 4 8 12]> [ 1 5 9 13]> [ 2 6 10 14]> [ 3 7 11 15]> [16 20 24 28]> [17 21 25 29]> [18 22 26 30]> [19 23 27 31]]> > and> > [[[ 0 4 8 12]> [ 1 5 9 13]]> > [[ 2 6 10 14]> [ 3 7 11 15]]> > [[16 20 24 28]> [17 21 25 29]]> > [[18 22 26 30]> [19 23 27 31]]]> > > > >> > -M> >> >> >> >> >> > ------------------------------------------------------------------------> >> > > Date: Sat, 17 May 2008 08:58:08 -0700> > > From: gherron [at] islandtraining> > > To: marlin_rowley [at] hotmail> > > CC: python-list [at] python> > > Subject: Re: numpy.frombuffer != unpack() ??> > >> > > Marlin Rowley wrote:> > > >> > > > Very cool.> > > >> > > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > > a represents a tile with height of 2 and width of 4 with 4 bits/pixel> > > > for each color.> > > >> > > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')> > > > this seperates the stream into individual values - Check> > > >> > > > > >>> b.shape=(2,4,4)> > > >> > > > This reshapes the array so that b.shape=(height,width,#bits/pixel) > > - Check> > > >> > > > >>> c = b.transpose((2,0,1))> > > >> > > > What does the (2,0,1) represent in terms of width and height and> > > > number of bytes?> > >> > > The (2,0,1) tells how to exchange the axes. For instance in a 2D array,> > > a normal transpose exchanges rows and columns. It will change a (a by> > > b) sized array into a (b by a) sized array. This would be equivalent to> > > the more saying interchange axes 0,1 to the new order of 1,0.> > >> > > In numpy with higher dimension arrays, the default transpose just> > > exchanges the first two axes, and the full transpose allows you to> > > specify exactly the new ordering of the exes.> > >> > > So transpose((2,0,1)) means take axes (0,1,2) to the new order> > > (2,1,0). In terms of sizes, an (a by b by c) sized array will end being> > > of size (c by a by b) in size.> > >> > > In terms of implementation, there may not be *any* data re-arrangement> > > in a transpose. The only thing that needs changing is how the indices> > > are converted to an actual machine address of an indexed item. The> > > documentation notes this by saying transpose returns a "new view" of > > the> > > array. This explains why I copied the array before extracting bytes> > > out of it -- you really do need the elements in the new order for the> > > next operation.> > >> > > Gary Herron> > >> > > >> > > >> > > >> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > >> > > > > Date: Fri, 16 May 2008 17:08:20 -0700> > > > > From: gherron [at] islandtraining> > > > > To: marlin_rowley [at] hotmail; python-list [at] python> > > > > Subject: Re: numpy.frombuffer != unpack() ??> > > > >> > > > > Marlin Rowley wrote:> > > > > > All:> > > > > >> > > > > > Say I have an array:> > > > > >> > > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > > > >> > > > > > How do I make it so that I now have:> > > > > >> > > > > > starting with first element (a[0])> > > > > > new_arr[0] = 'r'> > > > > > new_arr[1] = 'g'> > > > > > new_arr[2] = 'b'> > > > > > new_arr[3] = 'a'> > > > > > new_arr[4] = 'r'> > > > > > .....> > > > > >> > > > > > continuing "through" a[1] with the same new_arr> > > > > > new_arr[N] = 'r'> > > > > > new_arr[N+1] = 'g'> > > > > > ....> > > > > >> > > > > > -M> > > > >> > > > > Numpy can do this for you. First, do you really mean the array to> > > > > contain lists of one string each? If so:> > > > >> > > > > >>> import numpy> > > > > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])> > > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a> > > > > kludge here> > > > > >>> b> > > > > array([.'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', > > 'b', 'a',> > > > > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',> > > > > 'b', 'b', 'a', 'a', 'a', 'a'],> > > > > dtype='|S1')> > > > > >>> b.shape=(2,4,4)> > > > > >>> b> > > > > array([[['r', 'r', 'r', 'r'],> > > > > ['g', 'g', 'g', 'g'],> > > > > ['b', 'b', 'b', 'b'],> > > > > ['a', 'a', 'a', 'a']],> > > > >> > > > > [['r', 'r', 'r', 'r'],> > > > > ['g', 'g', 'g', 'g'],> > > > > ['b', 'b', 'b', 'b'],> > > > > ['a', 'a', 'a', 'a']]],> > > > > dtype='|S1')> > > > > >>> c = b.transpose((2,0,1))> > > > > >>> c> > > > > array([[['r', 'g', 'b', 'a'],> > > > > ['r', 'g', 'b', 'a']],> > > > >> > > > > [['r', 'g', 'b', 'a'],> > > > > ['r', 'g', 'b', 'a']],> > > > >> > > > > [['r', 'g', 'b', 'a'],> > > > > ['r', 'g', 'b', 'a']],> > > > >> > > > > [['r', 'g', 'b', 'a'],> > > > > ['r', 'g', 'b', 'a']]],> > > > > dtype='|S1')> > > > > >>> d=c.copy() # To make it contiguous> > > > > >>> d.shape = (32,)> > > > > >>> d> > > > > array([.'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', > > 'a', 'r',> > > > > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',> > > > > 'b', 'a', 'r', 'g', 'b', 'a'],> > > > > dtype='|S1')> > > > >> > > > > Done. Cool no?> > > > >> > > > > Gary Herron> > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > ------------------------------------------------------------------------> > > > > > From: marlin_rowley [at] hotmail> > > > > > To: robert.kern [at] gmail; python-list [at] python> > > > > > Subject: RE: numpy.frombuffer != unpack() ??> > > > > > Date: Fri, 16 May 2008 17:31:30 -0500> > > > > >> > > > > > Thank you! That solved it!> > > > > >> > > > > > -M> > > > > >> > > > > >> > > > > >> > > > > > ------------------------------------------------------------------------> > > > > >> > > > > > > To: python-list [at] python> > > > > > > From: robert.kern [at] gmail> > > > > > > Subject: Re: numpy.frombuffer != unpack() ??> > > > > > > Date: Fri, 16 May 2008 17:25:00 -0500> > > > > > >> > > > > > > Marlin Rowley wrote:> > > > > > > > All:> > > > > > > >> > > > > > > > I'm getting different floating point values when I use numpy> > > > > > vs. unpack().> > > > > > > >> > > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)> > > > > > > > buffer = unpack("!f", byte)> > > > > > > >> > > > > > > > frgba[0] != buffer[0]> > > > > > > >> > > > > > > > why? This is forcing me use the unpack() function since it's> > > > > > giving me> > > > > > > > the correct values. What am I doing wrong?> > > > > > >> > > > > > > Endianness, perhaps? '!' specifies big-endian data (an alias for> > > > > > '>'). Most> > > > > > > likely, you are on a little-endian platform. All of the dtypes> > > > > > in numpy default> > > > > > > to the native-endianness unless specified. If you want to read> > > > > > big-endian data> > > > > > > using numpy, do this:> > > > > > >> > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')> > > > > > >> > > > > > > If you have any more problems with numpy, please join us on the> > > > > > numpy mailing> > > > > > > list. When reporting problems, please try to provide a small but> > > > > > complete> > > > > > > snippet of self-contained code, the output that you got, and> > > > > > explain the output> > > > > > > that you expected to get. Thank you.> > > > > > >> > > > > > > http://www.scipy.org/Mailing_Lists> > > > > > >> > > > > > > --> > > > > > > Robert Kern> > > > > > >> > > > > > > "I have come to believe that the whole world is an enigma, a> > > > > > harmless enigma> > > > > > > that is made terrible by our own mad attempt to interpret it as> > > > > > though it had> > > > > > > an underlying truth."> > > > > > > -- Umberto Eco> > > > > > >> > > > > > > --> > > > > > > http://mail.python.org/mailman/listinfo/python-list> > > > > >> > > > > >> > > > > >> > > > > > ------------------------------------------------------------------------> > > > > > E-mail for the greater good. Join the i’m Initiative from> > > > > > Microsoft.> > > > > >> > > > > > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > ------------------------------------------------------------------------> > > > > > E-mail for the greater good. Join the i’m Initiative from > > Microsoft.> > > > > >> > > > > > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood> > >> > > >> > > > > >> > > > > >> > > > > > ------------------------------------------------------------------------> > > > > >> > > > > > --> > > > > > http://mail.python.org/mailman/listinfo/python-list> > > > >> > > >> > > >> > > > > > ------------------------------------------------------------------------> > > > Keep your kids safer online with Windows Live Family Safety. Help> > > > protect your kids.> > > > > > <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> > >> > > >> > > > > > ------------------------------------------------------------------------> > > >> > > > --> > > > http://mail.python.org/mailman/listinfo/python-list> > >> >> >> > ------------------------------------------------------------------------> > Keep your kids safer online with Windows Live Family Safety. Help > > protect your kids. > > <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> > >> > ------------------------------------------------------------------------> >> > --> > http://mail.python.org/mailman/listinfo/python-list>
_________________________________________________________________
E-mail for the greater good. Join the i’m Initiative from Microsoft.
http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_ GreaterGood

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads