Mailing List Archive: Python: Python

no data exclution and unique combination.

Index | Next | Previous | View Threaded

giuseppe.amatulli at gmail

Jul 24, 2012, 11:27 AM

Post #1 of 9 (231 views)
Permalink

no data exclution and unique combination.

Hi,
would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.

given

a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])

no_data_a=1
no_data_b=2

a_clean=array([4,4,5,4,4,4])
b_clean=array([3,5,4,4,3,4])

after i need to calculate unique combination in pairs to count the observations
and obtain
(4,3,2)
(4,5,1)
(5,4,1)
(4,4,2)

For the fist task i did

a_No_data_a = a[a != no_data_a]
b_No_data_a = b[a != no_data_a]

b_clean = b_No_data_a[b_No_data_a != no_data_b]
a_clean = a_No_data_a[a_No_data_a != no_data_b]

but the results are not really stable.

For the second task
The np.unique would solve the problem if it can be apply to a two arrays.

Any idea?
thanks in advance
Giuseppe

--
http://mail.python.org/mailman/listinfo/python-list

python at mrabarnett

Jul 24, 2012, 11:51 AM

Post #2 of 9 (229 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 24/07/2012 19:27, giuseppe.amatulli [at] gmail wrote:
> Hi,
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>
> given
>
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>
> no_data_a=1
> no_data_b=2
>
> a_clean=array([4,4,5,4,4,4])
> b_clean=array([3,5,4,4,3,4])
>
> after i need to calculate unique combination in pairs to count the observations
> and obtain
> (4,3,2)
> (4,5,1)
> (5,4,1)
> (4,4,2)
>
> For the fist task i did
>
> a_No_data_a = a[a != no_data_a]
> b_No_data_a = b[a != no_data_a]
>
> b_clean = b_No_data_a[b_No_data_a != no_data_b]
> a_clean = a_No_data_a[a_No_data_a != no_data_b]
>
> but the results are not really stable.
>
mask = (a != no_data_a) & (b != no_data_b)
a_clean = a[mask]
b_clean = b[mask]

> For the second task
> The np.unique would solve the problem if it can be apply to a two arrays.
>

--
http://mail.python.org/mailman/listinfo/python-list

python at mrabarnett

Jul 24, 2012, 12:08 PM

Post #3 of 9 (234 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 24/07/2012 19:51, MRAB wrote:
> On 24/07/2012 19:27, giuseppe.amatulli [at] gmail wrote:
>> Hi,
>> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>>
>> given
>>
>> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
>> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>>
>> no_data_a=1
>> no_data_b=2
>>
>> a_clean=array([4,4,5,4,4,4])
>> b_clean=array([3,5,4,4,3,4])
>>
>> after i need to calculate unique combination in pairs to count the observations
>> and obtain
>> (4,3,2)
>> (4,5,1)
>> (5,4,1)
>> (4,4,2)
>>
>> For the fist task i did
>>
>> a_No_data_a = a[a != no_data_a]
>> b_No_data_a = b[a != no_data_a]
>>
>> b_clean = b_No_data_a[b_No_data_a != no_data_b]
>> a_clean = a_No_data_a[a_No_data_a != no_data_b]
>>
>> but the results are not really stable.
>>
> mask = (a != no_data_a) & (b != no_data_b)
> a_clean = a[mask]
> b_clean = b[mask]
>
>> For the second task
>> The np.unique would solve the problem if it can be apply to a two arrays.
>>
I couldn't figure out how to do the second part in numpy, so:

from collections import Counter
counts = Counter(zip(a_clean, b_clean))
counts = [pair + (count,) for pair, count in counts.items()]

--
http://mail.python.org/mailman/listinfo/python-list

ian.g.kelly at gmail

Jul 24, 2012, 12:28 PM

Post #4 of 9 (223 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On Jul 24, 2012 12:32 PM, <giuseppe.amatulli [at] gmail> wrote:

> after i need to calculate unique combination in pairs to count the
observations
> and obtain
> (4,3,2)
> (4,5,1)
> (5,4,1)
> (4,4,2)

I don't know about a numpy solution, but this could be achieved by
collections.Counter(zip(a, b)).items(). That gives you:
((4,3),2)
((4,5),1)
Etc.

If you really want triples instead of pair-value pairs, then you would need
to flatten the tuples yourself.

tjreedy at udel

Jul 24, 2012, 12:32 PM

Post #5 of 9 (226 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 7/24/2012 2:27 PM, giuseppe.amatulli [at] gmail wrote:
> Hi,
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>
> given
>
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>
> no_data_a=1
> no_data_b=2
>
> a_clean=array([4,4,5,4,4,4])
> b_clean=array([3,5,4,4,3,4])

As I discovered when running the solution before, your test data are
wrong, leaving out 2,3 before the last pair (4,4). Anyway, for those
interested in a plain Python solution, without numpy:

a=[1,2,4,4,5,4,1,4,1,1,2,4]
b=[1,2,3,5,4,4,1,3,2,1,3,4]

no_data_a=1
no_data_b=2

a_clean=(4,4,5,4,4,2,4)
b_clean=(3,5,4,4,3,3,4)

cleaned = list(zip(*(pair for pair in zip(a,b)
if pair[0] != no_data_a and pair[1] != no_data_b)))
print(cleaned, cleaned == [a_clean, b_clean])
#
[.(4, 4, 5, 4, 4, 2, 4), (3, 5, 4, 4, 3, 3, 4)] True

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

giuseppe.amatulli at gmail

Aug 9, 2012, 1:06 PM

Post #6 of 9 (191 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

Terry and MRAB,
thanks for yours suggestions,
in the end i found this solution

mask=( a != 0 ) & ( b != 0 )

a_mask=a[mask]
b_mask=b[mask]

array2D = np.array(zip(a_mask,b_mask))

unique=dict()
for row in array2D :
row = tuple(row)
if row in unique:
unique[row] += 1
else:
unique[row] = 1

print unique
{(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}

I choose this solution because i could not install "from collections import Counter".
Anyway how i can print to a file the unique results without the brackets and obtain something like this?
4 5 1
5 4 1
4 4 2
2 3 1
4 3 2

Thanks in advance
Best regards.
Giuseppe

On Tuesday, 24 July 2012 13:27:05 UTC-5, giuseppe...@gmail.com wrote:
> Hi,
>
> would like to take eliminate a specific number in an array and its correspondent in an other array, and vice-versa.
>
>
>
> given
>
>
>
> a=np.array([1,2,4,4,5,4,1,4,1,1,2,4])
>
> b=np.array([1,2,3,5,4,4,1,3,2,1,3,4])
>
>
>
> no_data_a=1
>
> no_data_b=2
>
>
>
> a_clean=array([4,4,5,4,4,4])
>
> b_clean=array([3,5,4,4,3,4])
>
>
>
> after i need to calculate unique combination in pairs to count the observations
>
> and obtain
>
> (4,3,2)
>
> (4,5,1)
>
> (5,4,1)
>
> (4,4,2)
>
>
>
> For the fist task i did
>
>
>
> a_No_data_a = a[a != no_data_a]
>
> b_No_data_a = b[a != no_data_a]
>
>
>
> b_clean = b_No_data_a[b_No_data_a != no_data_b]
>
> a_clean = a_No_data_a[a_No_data_a != no_data_b]
>
>
>
> but the results are not really stable.
>
>
>
> For the second task
>
> The np.unique would solve the problem if it can be apply to a two arrays.
>
>
>
> Any idea?
>
> thanks in advance
>
> Giuseppe

--
http://mail.python.org/mailman/listinfo/python-list

d at davea

Aug 9, 2012, 2:23 PM

Post #7 of 9 (190 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 08/09/2012 04:06 PM, giuseppe.amatulli [at] gmail wrote:
> <SNIP>
>
> print unique
> {(4, 5): 1, (5, 4): 1, (4, 4): 2, (2, 3): 1, (4, 3): 2}
>
> I choose this solution because i could not install "from collections import Counter".

Nothing to install, at least for Python 2.7. collections is in the
standard library, and Counter is in collections (new in version 2.7)

> Anyway how i can print to a file the unique results without the brackets and obtain something like this?
> 4 5 1
> 5 4 1
> 4 4 2
> 2 3 1
> 4 3 2
>
>

To print out a dict in an explicit format, you want a loop.

for key, val in unique.items():
print key[0], key[1], val

Note that you cannot guarantee the order they will print out, once
stored in a dict. You may want to sort them, or otherwise order them if
it matters.

<SNIP>

Note I added my responses after the parts of your message that I was
quoting. To put the response first is called top-posting, and against
policy here.

--

DaveA

--
http://mail.python.org/mailman/listinfo/python-list

tjreedy at udel

Aug 9, 2012, 2:33 PM

Post #8 of 9 (190 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 8/9/2012 4:06 PM, giuseppe.amatulli [at] gmail wrote:
> Terry and MRAB,
> thanks for yours suggestions,
> in the end i found this solution
>
>
> mask=( a != 0 ) & ( b != 0 )
>
> a_mask=a[mask]
> b_mask=b[mask]
>
> array2D = np.array(zip(a_mask,b_mask))
>
> unique=dict()
> for row in array2D :
> row = tuple(row)
> if row in unique:
> unique[row] += 1
> else:
> unique[row] = 1

I believe the 4 lines above are equivalent to
unique[row] = unique.get(row, 0) + 1

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

hansmu at xs4all

Aug 10, 2012, 1:47 AM

Post #9 of 9 (186 views)
Permalink

Re: no data exclution and unique combination. [In reply to]

On 9/08/12 23:33:58, Terry Reedy wrote:
> On 8/9/2012 4:06 PM, giuseppe.amatulli [at] gmail wrote:
[...]
>> unique=dict()
>> for row in array2D :
>> row = tuple(row)
>> if row in unique:
>> unique[row] += 1
>> else:
>> unique[row] = 1
>
> I believe the 4 lines above are equivalent to
> unique[row] = unique.get(row, 0) + 1

I would write that bit as:

from collections import defaultdict

unique = defaultdict(int)
for row in array2D:
unique[row] += 1

Hope this helps,

-- HansM
--
http://mail.python.org/mailman/listinfo/python-list

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads