Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

Making the Lua/Javascript decision (Re: Performance roadmap update)

 

 

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


robla at wikimedia

Dec 20, 2011, 11:33 PM

Post #1 of 12 (637 views)
Permalink
Making the Lua/Javascript decision (Re: Performance roadmap update)

On Wed, Dec 14, 2011 at 9:00 PM, Tim Starling <tstarling [at] wikimedia> wrote:
> We still want to do something about parser performance in the first
> half of 2012, so we're going to bring forward our other performance
> project, i.e. server-side scripting embedded in wikitext. That's a
> project which is still at an early stage of planning. We will need to
> define its scope, and to bite the bullet and make some tough design
> choices (such as Lua versus JavaScript), if it's going to progress
> from pipe dream to reality.

Let's resolve to have all of the big pieces nailed down by January 31.
In particular, it should be possible to make the WikiScript vs Lua vs
Javascript decision well before that time.

There already seems to be a credible starting point for Lua and
WikiScript, but not yet one for Javascript or any other language. Is
there anyone willing to put together a Javascript proof-of-concept?

Rob

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


owen at wikia-inc

Dec 22, 2011, 11:29 AM

Post #2 of 12 (589 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

Yep, this is for server side code, giving users a better template programming language.

On Dec 22, 2011, at 7:17 AM, Jay Ashworth wrote:

> ----- Original Message -----
>> From: "Peter Kaminski" <kaminski [at] istori>
>
>> I don't have anything particular against Lua and no particular love for
>> JavaScript, but JS seems more web-native; plus code and skills developed
>> can accrue to both client- and server-side of the web.
>
> Just so I'm clear: this is a server-side question?
>
> Cause if it's client side, "JS is already in the browser" seems pretty
> compelling to me...
>
> Cheers,
> -- jra
> --
> Jay R. Ashworth Baylink jra [at] baylink
> Designer The Things I Think RFC 2100
> Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII
> St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


owen at wikia-inc

Dec 22, 2011, 11:50 AM

Post #3 of 12 (593 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

I think that would be a nice feature to have. Amazingly enough, WikiScripts is not of the extensions that we have in our codebase at wikia (I think we're over 800+), but I will take a look at the implementation. The <lua> parser hook allows for parameters, but that's not the same. Having access to normal template parameters would allow a lua template to act as a drop in replacement for an existing template, which is essential.

We are in a code freeze period here @ wikia, but when we return from the holidays in January I can get the Lua extension enabled on a test wiki (lua.wikia.com) so that other people can experiment with this.

On Dec 21, 2011, at 9:29 PM, Dmitriy Sintsov wrote:

> * Owen Davis <owen [at] wikia-inc> [Wed, 21 Dec 2011 19:18:46 +0000
> (UTC)]:
>> It is littered with embedded HTML and string.format statements. Ugh.
>> I'm going to look at something simple like Moustache (which has both
>> JS and Lua implementations already) as a proof of concept. Does
>> anybody have a better suggestion? I think Lua (or JS) by itself is
> not
>> enough, there has to be a nice library of utility functions. Has
> anyone
>> thought about what that will really look like?
>>
> Also it would be great if Lua binding could perform parser functions. I
> believe that Extension:WikiScripts uses parser frame and can use current
> template parameters, when available:
> http://www.mediawiki.org/wiki/Extension:WikiScripts
> Template library (tpl_) allows the script to access the parser and the
> parameters of the template that invokes the script.
>
> tpl_arg( argname[, default] ) returns the argument which name is
> specified as an argument, or the default value if it is not set (if no
> default is specified, it returns false).
> tpl_named_args() returns all the named arguments to the template.
> tpl_numbered_args() returns all the numbered arguments.
> tpl_is_transcluded() returns whether the code is invoked from a
> template.
>
> Dmitriy
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


questpc at rambler

Dec 22, 2011, 9:26 PM

Post #4 of 12 (598 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

* Owen Davis <owen [at] wikia-inc> [Thu, 22 Dec 2011 11:50:48 -0800]:
> I think that would be a nice feature to have. Amazingly enough,
> WikiScripts is not of the extensions that we have in our codebase at
> wikia (I think we're over 800+), but I will take a look at the
> implementation. The <lua> parser hook allows for parameters, but
that's
> not the same. Having access to normal template parameters would allow
a
> lua template to act as a drop in replacement for an existing template,
> which is essential.
>
WikiScripts parses and executes it's scripts via PHP, which has both
advantages and disadvantages. Lua extension should run faster, however
it is not suitable for every hosting out there, because PHP Lua module
is uncommon. I wonder whether it's possible to run around without PHP
Lua extension, perhaps to daemonize Lua then exchange data between
apache/mod-php and lua processes via named pipes? It is really sad that
PHP itself does not have stable "VM" allowing to execute safe subset of
it's own code in server-side scripts (it had only some beta or alpha
status module). I wonder whether it is possible to cross-translate the
subset of WikiScripts or Lua into PHP source, then PHP eval it (however
would that work with HipHop?).

Also it would be great if WikiScripts or Lua extension allowed to easily
bind functions / class wrappers for another MediaWiki extensions. The
extension I develop uses PHP eval for interpretation of the scripts, I
am looking for adaption of WikiScripts to use it in conjunction with my
extension.
Dmitriy

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


vasilvv at gmail

Dec 22, 2011, 9:46 PM

Post #5 of 12 (588 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

On Fri, Dec 23, 2011 at 9:26 AM, Dmitriy Sintsov <questpc [at] rambler> wrote:
> Also it would be great if WikiScripts or Lua extension allowed to easily
> bind functions / class wrappers for another MediaWiki extensions. The
> extension I develop uses PHP eval for interpretation of the scripts, I
> am looking for adaption of WikiScripts to use it in conjunction with my
> extension.
> Dmitriy

Well, WikiScripts, if we decide to develop it seriously, will
certainly have extension interface (it is on my large TODO list for
that). Something tells me it would be possible to do in Lua as well.
All we have to do now is to settle down on what way we choose.

--vvv

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


questpc at rambler

Dec 22, 2011, 9:54 PM

Post #6 of 12 (586 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

* Victor Vasiliev <vasilvv [at] gmail> [Fri, 23 Dec 2011 09:46:46 +0400]:
> On Fri, Dec 23, 2011 at 9:26 AM, Dmitriy Sintsov <questpc [at] rambler>
> wrote:
> > Also it would be great if WikiScripts or Lua extension allowed to
> easily
> > bind functions / class wrappers for another MediaWiki extensions.
The
> > extension I develop uses PHP eval for interpretation of the scripts,
I
> > am looking for adaption of WikiScripts to use it in conjunction with
> my
> > extension.
> > Dmitriy
>
> Well, WikiScripts, if we decide to develop it seriously, will
> certainly have extension interface (it is on my large TODO list for
> that). Something tells me it would be possible to do in Lua as well.
> All we have to do now is to settle down on what way we choose.
>
Victor, what do you think about making WikiScripts syntax more similar
to the subset of Lua or JavaScript syntax? That probably should not be
too hard? What's about cross-translating to PHP then eval()?
Lua is great, however, it's a bit strange to use two interpreters
(PHP+Lua) together. That limits hosting possibilities and it's something
like using two similar screwdrivers for the same screw. It is a shame
PHP itself does not have stable good VM extension for execution of
scripts.
Dmitriy

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


vasilvv at gmail

Dec 22, 2011, 11:05 PM

Post #7 of 12 (600 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

On Fri, Dec 23, 2011 at 9:54 AM, Dmitriy Sintsov <questpc [at] rambler> wrote:
> Victor, what do you think about making WikiScripts syntax more similar
> to the subset of Lua or JavaScript syntax? That probably should not be
> too hard?
It is already almost a JavaScript subset

> What's about cross-translating to PHP then eval()?
Bad idea. If you really care about performance, you will be able to
install native parser
(already implemented, but not integrated) and that shall solve most
performance problems.

> Lua is great, however, it's a bit strange to use two interpreters
> (PHP+Lua) together. That limits hosting possibilities and it's something
> like using two similar screwdrivers for the same screw.
Not really. Lua was designed as a sandboxable language, and PHP was not.

--vvv

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jra at baylink

Dec 23, 2011, 6:30 AM

Post #8 of 12 (596 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

----- Original Message -----
> From: "Victor Vasiliev" <vasilvv [at] gmail>

> > Lua is great, however, it's a bit strange to use two interpreters
> > (PHP+Lua) together. That limits hosting possibilities and it's
> > something
> > like using two similar screwdrivers for the same screw.

> Not really. Lua was designed as a sandboxable language, and PHP was
> not.

This is a really critical point: if you're going to provide an interpreted
language to end-users from within a program that is, itself, written in
an interpreted language, *you cannot use the underlying interpreter* to
run the end-users' programs, unless that interpreter has sandboxing built-in.

If you try, you will almost certainly be exposing yourself to critical
security vulnerabilities. You're almost *better* off picking a different
language, so that you're not tempted to try.

Cheers,
-- jra
--
Jay R. Ashworth Baylink jra [at] baylink
Designer The Things I Think RFC 2100
Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


questpc at rambler

Dec 23, 2011, 9:21 AM

Post #9 of 12 (582 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

On 23.12.2011 18:30, Jay Ashworth wrote:
> This is a really critical point: if you're going to provide an
> interpreted language to end-users from within a program that is,
> itself, written in an interpreted language, *you cannot use the
> underlying interpreter* to run the end-users' programs, unless that
> interpreter has sandboxing built-in. If you try, you will almost
> certainly be exposing yourself to critical security vulnerabilities.
> You're almost *better* off picking a different language, so that
> you're not tempted to try. Cheers, -- jra
I remember that PHP had some outdated and unmaintained sandboxing PECL
module, however it's unmaintained for a long time.
http://php.net/manual/en/runkit.sandbox.php
Dmitriy


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


keisial at gmail

Dec 25, 2011, 2:16 PM

Post #10 of 12 (585 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

On 23.12.2011 18:21, Dmitriy Sintsov wrote:
> I remember that PHP had some outdated and unmaintained sandboxing PECL
> module, however it's unmaintained for a long time.
> http://php.net/manual/en/runkit.sandbox.php
> Dmitriy

Dmitry Zenovich applied for maintaining it last year, although little
seems to have been done in the official repo. He seems to have been
working on https://github.com/zenovich/runkit/
I worked with runkit some years ago and it wasn't hard to make it run.
The downside is that you need a threaded php.

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


questpc at rambler

Dec 25, 2011, 10:08 PM

Post #11 of 12 (588 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

* Keisial <keisial [at] gmail> [Sun, 25 Dec 2011 23:16:46 +0100]:
> On 23.12.2011 18:21, Dmitriy Sintsov wrote:
> > I remember that PHP had some outdated and unmaintained sandboxing
PECL
> > module, however it's unmaintained for a long time.
> > http://php.net/manual/en/runkit.sandbox.php
> > Dmitriy
>
> Dmitry Zenovich applied for maintaining it last year, although little
> seems to have been done in the official repo. He seems to have been
> working on https://github.com/zenovich/runkit/
> I worked with runkit some years ago and it wasn't hard to make it run.
> The downside is that you need a threaded php.
>
Wikimedia has experienced PHP/C developers who probably can maintain and
update such module. Maybe even cross-translation from subset of JS into
PHP (with caching) then executing via runkit could be possible. But
threading is not something that is desirable in current environment. So
maybe it was a bad idea.
Dmitriy

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tparscal at wikimedia

Jan 9, 2012, 10:51 AM

Post #12 of 12 (523 views)
Permalink
Re: Making the Lua/Javascript decision (Re: Performance roadmap update) [In reply to]

I was working with Victor a bit to see how close to JS we could get
WikiScripts syntax to be, and it seemed like it was possible but some
things (like objects) would either be a hack to "fake" in some ways or a
larger job to get working. Not sure if he's got more progress in that way
to show, but subsets are good because you can always add more later. As
long as we don't add support for something that's not standard javascript
(super-set) we have a lot of options in the future. Web standards are
important - we should strive to use them as much as possible.

- Trevor

On Sun, Dec 25, 2011 at 10:08 PM, Dmitriy Sintsov <questpc [at] rambler>wrote:

> * Keisial <keisial [at] gmail> [Sun, 25 Dec 2011 23:16:46 +0100]:
> > On 23.12.2011 18:21, Dmitriy Sintsov wrote:
> > > I remember that PHP had some outdated and unmaintained sandboxing
> PECL
> > > module, however it's unmaintained for a long time.
> > > http://php.net/manual/en/runkit.sandbox.php
> > > Dmitriy
> >
> > Dmitry Zenovich applied for maintaining it last year, although little
> > seems to have been done in the official repo. He seems to have been
> > working on https://github.com/zenovich/runkit/
> > I worked with runkit some years ago and it wasn't hard to make it run.
> > The downside is that you need a threaded php.
> >
> Wikimedia has experienced PHP/C developers who probably can maintain and
> update such module. Maybe even cross-translation from subset of JS into
> PHP (with caching) then executing via runkit could be possible. But
> threading is not something that is desirable in current environment. So
> maybe it was a bad idea.
> Dmitriy
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.