Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

faster than list.extend()

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


hyunchul.mailing at gmail

Nov 16, 2009, 2:30 PM

Post #1 of 8 (359 views)
Permalink
faster than list.extend()

Hi, all.

I want to improve speed of following simple function.
Any suggestion?

**********
def triple(inputlist):
results = []
for x in inputlist:
results.extend([x,x,x])
return results
**********

Thank you in advance,

Hyunchul


clp2 at rebertia

Nov 16, 2009, 2:41 PM

Post #2 of 8 (346 views)
Permalink
Re: faster than list.extend() [In reply to]

On Mon, Nov 16, 2009 at 2:30 PM, Hyunchul Kim
<hyunchul.mailing [at] gmail> wrote:
> Hi, all.
>
> I want to improve speed of following simple function.
> Any suggestion?
>
> **********
> def triple(inputlist):
>   results = []
>   for x in inputlist:
>     results.extend([x,x,x])
>   return results
> **********

You'd probably be better off calling .append() 3 times instead of
.extend() as it would avoid the creation of all those intermediate
3-element lists.

Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list


python.list at tim

Nov 16, 2009, 2:55 PM

Post #3 of 8 (343 views)
Permalink
Re: faster than list.extend() [In reply to]

Hyunchul Kim wrote:
> Hi, all.
>
> I want to improve speed of following simple function.
> Any suggestion?
>
> **********
> def triple(inputlist):
> results = []
> for x in inputlist:
> results.extend([x,x,x])
> return results
> **********

Several ways occur to me:

def t1(i):
for x in i:
yield x
yield x
yield x

def t2(i):
r = range(3)
return (x for x in i for _ in r)


I prefer to return generators in case the input is large, but in
both cases, you can just wrap it in list() like

list(t1(inputlist))

or in t2(), you can just change it to use

return [x for x in i for _ in r]

-tkc




--
http://mail.python.org/mailman/listinfo/python-list


python at rcn

Nov 16, 2009, 3:28 PM

Post #4 of 8 (343 views)
Permalink
Re: faster than list.extend() [In reply to]

On Nov 16, 2:41 pm, Chris Rebert <c...@rebertia.com> wrote:
> On Mon, Nov 16, 2009 at 2:30 PM, Hyunchul Kim
>
> <hyunchul.mail...@gmail.com> wrote:
> > Hi, all.
>
> > I want to improve speed of following simple function.
> > Any suggestion?
>
> > **********
> > def triple(inputlist):
> >   results = []
> >   for x in inputlist:
> >     results.extend([x,x,x])
> >   return results


Here's an itertools variant that is somewhat speedy when the inputlist
is long:

from itertools import *

def triple3(inputlist, list=list,
chain_from_iterable=chain.from_iterable, izip=izip):
return list(chain_from_iterable(izip(inputlist, inputlist,
inputlist)))

Raymond


--
http://mail.python.org/mailman/listinfo/python-list


gagsl-py2 at yahoo

Nov 16, 2009, 6:15 PM

Post #5 of 8 (337 views)
Permalink
Re: faster than list.extend() [In reply to]

En Mon, 16 Nov 2009 19:30:27 -0300, Hyunchul Kim
<hyunchul.mailing [at] gmail> escribió:

> I want to improve speed of following simple function.
> Any suggestion?
>
> **********
> def triple(inputlist):
> results = []
> for x in inputlist:
> results.extend([x,x,x])
> return results
> **********

These are my best attempts:

def triple3(inputlist):
results = []
append = results.append
for x in inputlist:
append(x); append(x); append(x)
return results

def triple4(inputlist, _three=xrange(3)):
return [x for x in inputlist for _ in _three]

For a 400-items list, triple3 is 40% faster and triple4 25% faster than
yours.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list


jasonsewall at gmail

Nov 16, 2009, 6:56 PM

Post #6 of 8 (333 views)
Permalink
Re: faster than list.extend() [In reply to]

On Mon, Nov 16, 2009 at 6:28 PM, Raymond Hettinger <python [at] rcn> wrote:
> On Nov 16, 2:41 pm, Chris Rebert <c...@rebertia.com> wrote:
>> On Mon, Nov 16, 2009 at 2:30 PM, Hyunchul Kim
>>
>> <hyunchul.mail...@gmail.com> wrote:
>> > Hi, all.
>>
>> > I want to improve speed of following simple function.
>> > Any suggestion?
>>
>> > **********
>> > def triple(inputlist):
>> >   results = []
>> >   for x in inputlist:
>> >     results.extend([x,x,x])
>> >   return results
>
>
> Here's an itertools variant that is somewhat speedy when the inputlist
> is long:
>
>    from itertools import *
>
>    def triple3(inputlist, list=list,
> chain_from_iterable=chain.from_iterable, izip=izip):
>        return list(chain_from_iterable(izip(inputlist, inputlist,
> inputlist)))

Even (slightly) faster:

def triple3_mk2(inputlist):
return list(chain.from_iterable(izip(*repeat(inputlist, 3))))

I tried something like this when I saw Hyunchul's original email, but
I didn't know about chain.from_iterable and it was pretty slow
compared to the original. Adding (itertools.)repeat to Raymonds
speeds it up 5-10% more.

I'm pretty surprised this is faster than Tim's t2 (which is actually a
little slower than the original); I always thought of comprehensions
as the fastest way to do this find of stuff.

Jason
--
http://mail.python.org/mailman/listinfo/python-list


hyunchul.mailing at gmail

Nov 17, 2009, 12:07 AM

Post #7 of 8 (331 views)
Permalink
Re: faster than list.extend() [In reply to]

Thank Tim, Raymond, and all.

In my test, Tim's t1 was fastest and Raymond's triple3 is very near to Tim's
t1.
Anyway both of them took only 6~7% time of my original .extend() version.

I think that following n_ple() is a good generalized function that was 1~3%
slower than t1() and triple3() for "triple".

*********
from itertools import *
def n_ple(inputlist, n):
chain_from_iterable = chain.from_iterable; izip = izip
return chain_from_iterable(izip(*[inputlist]*n))
*********

def t2(i):
r = range(3)
return (x for x in i for _ in r)
def t2_1(i):
r = range(3)
return [x for x in i for _ in r]

I didn't know that list(t2(inputlist)) is much faster than t2_1(inputlist),
it's gonna be a helpful tip for me.

Thank you all.

Hyunchul


On Tue, Nov 17, 2009 at 7:55 AM, Tim Chase <python.list [at] tim>wrote:

> Hyunchul Kim wrote:
>
>> Hi, all.
>>
>> I want to improve speed of following simple function.
>> Any suggestion?
>>
>> **********
>> def triple(inputlist):
>> results = []
>> for x in inputlist:
>> results.extend([x,x,x])
>> return results
>> **********
>>
>
> Several ways occur to me:
>
> def t1(i):
> for x in i:
> yield x
> yield x
> yield x
>
> def t2(i):
> r = range(3)
> return (x for x in i for _ in r)
>
>
> I prefer to return generators in case the input is large, but in both
> cases, you can just wrap it in list() like
>
> list(t1(inputlist))
>
> or in t2(), you can just change it to use
>
> return [x for x in i for _ in r]
>
> -tkc
>
>
>
>
>


__peter__ at web

Nov 17, 2009, 2:24 AM

Post #8 of 8 (328 views)
Permalink
Re: faster than list.extend() [In reply to]

Gabriel Genellina wrote:

> En Mon, 16 Nov 2009 19:30:27 -0300, Hyunchul Kim
> <hyunchul.mailing [at] gmail> escribió:
>
>> I want to improve speed of following simple function.
>> Any suggestion?
>>
>> **********
>> def triple(inputlist):
>> results = []
>> for x in inputlist:
>> results.extend([x,x,x])
>> return results
>> **********
>
> These are my best attempts:
>
> def triple3(inputlist):
> results = []
> append = results.append
> for x in inputlist:
> append(x); append(x); append(x)
> return results
>
> def triple4(inputlist, _three=xrange(3)):
> return [x for x in inputlist for _ in _three]
>
> For a 400-items list, triple3 is 40% faster and triple4 25% faster than
> yours.

[I didn't see the original post]

If inputlist is actually a list or tuple the following should beat them all:

def triple_repeat(items):
result = [None] * (3*len(items))
result[::3] = result[1::3] = result[2::3] = items
return result

For small n the generalization should still be quite competitive:

def n_repeat(items, n):
items = tuple(items)
result = [None]*(len(items) * n)
for offset in range(n):
result[offset::n] = items
return result

Some measurements:

$ cat extend.py
from itertools import *

def triple_repeat(items):
result = [None] * (3*len(items))
result[::3] = result[1::3] = result[2::3] = items
return result

def n_repeat(items, n):
items = tuple(items)
result = [None]*(len(items) * n)
for offset in range(n):
result[offset::n] = items
return result

def t1(i):
for x in i:
yield x
yield x
yield x

def triple3(inputlist, list=list, chain_from_iterable=chain.from_iterable, izip=izip):
return list(chain_from_iterable(izip(inputlist, inputlist, inputlist)))

data = range(1000)


$ python -m timeit -s 'from extend import triple_repeat, data' 'triple_repeat(data)'
10000 loops, best of 3: 52.4 usec per loop
$ python -m timeit -s 'from extend import n_repeat, data' 'n_repeat(data, 3)'
10000 loops, best of 3: 60.7 usec per loop
$ python -m timeit -s 'from extend import t1, data' 'list(t1(data))'
1000 loops, best of 3: 337 usec per loop
$ python -m timeit -s 'from extend import triple3, data' 'triple3(data)'
1000 loops, best of 3: 231 usec per loop

Peter

--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.