Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Wikipedia: Wikitech

Moving forward with Lua

 

 

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded


robla at wikimedia

Jan 26, 2012, 5:44 PM

Post #1 of 9 (4833 views)
Permalink
Moving forward with Lua

Hi folks,

As you may know, WMF's Platform Engineering group plans to embark on a
major performance initiative this year, and had chosen inline
scripting as having the biggest potential impact given what's
practical now. Tim Starling build a Lua prototype last year which
showed a lot of promise for making things much faster. One major
decision before embarking on this effort was a decision on whether
we'd stick with Lua or try another language such as Javascript or
Victor's WikiScript implmentation. I wanted to make a decision by the
end of the month[1], and I think we've done it.

We've decided to build a deployable version of Lua as a new
alternative to wiki markup for templates, barring some scandalous
revelation about Lua's lurid past or other unforeseen barrier. Tim
will be leading this effort, and will start on the implementation some
time after the dust settles on the 1.19 deployment and the Git
migration. The project page for this is located here:
http://www.mediawiki.org/wiki/Lua_scripting

Rough notes from our meeting yesterday are also available [2]

Rob

[1] http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/57769/focus=57813
[2] http://www.mediawiki.org/wiki/Lua_scripting/Meeting_2012-01-25

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


Platonides at gmail

Jan 28, 2012, 2:51 PM

Post #2 of 9 (4685 views)
Permalink
Re: Moving forward with Lua [In reply to]

Doesn't seem to have arrived into the list. Resending.

On 27/01/12 16:17, Platonides wrote:
> 1) Is there a simple Lua syntax guide?
> http://www.lua.org/manual/5.2/ looks more like a grammar. Certainly
> not suitable for end users.
>
> 2) The decision seems to have been Lua vs JS. What discarded a custom
> language? (such as WikiScripts)
>
> 3) How hard is it to translate Lua to PHP?
> (for templates which then get converted into extensions)
>
> 4) Where would code live?
>
> 5) I think it _would_ be possible to make an extension defined with
> <wikiscript> to take over the page. (avoiding the need of template
> wrappers)
>
> 6) What's the expected way of implementing 'libraries' in the wiki?



_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


vasilvv at gmail

Jan 28, 2012, 3:39 PM

Post #3 of 9 (4693 views)
Permalink
Re: Moving forward with Lua [In reply to]

On Sun, Jan 29, 2012 at 2:51 AM, Platonides <Platonides [at] gmail> wrote:
>> 1) Is there a simple Lua syntax guide?
>> http://www.lua.org/manual/5.2/ looks more like a grammar. Certainly not
>> suitable for end users.

Wikipedia article has some nice examples.

>> 2) The decision seems to have been Lua vs JS. What discarded a custom
>> language? (such as WikiScripts)

As far as I am aware, those are mostly performance issues. Although
there may be maintaining issues as well.

>> 3) How hard is it to translate Lua to PHP?
>> (for templates which then get converted into extensions)

About as hard as JS -> PHP (since they are very similar in many questions).

>> 4) Where would code live?
>>
>> 5) I think it _would_ be possible to make an extension defined with
>> <wikiscript> to take over the page. (avoiding the need of template wrappers)
>>
>> 6) What's the expected way of implementing 'libraries' in the wiki?

That's sort of unsolved question. Check out the meeting notes for some
thoughts on that.

Answer to #4 is almost certainly separate pages as far as I
understand. This approach allows us to have syntax highlighting, code
editor and a better debugger. According to meeting notes, this may
also be problematic for visual editor.

The way WikiScripts extensions did it is that there are special code
pages which are called modules, and you can export stuff (functions
and constants and probably classes in Lua) from modules like you do in
CommonJS and call one module's methods from another's. I am now
working on the Scripting extension which is a language-independent
scripting module based on WikiScripts code:

http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/Scripting/

--vvv

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


sumanah at wikimedia

Feb 1, 2012, 10:22 AM

Post #4 of 9 (4638 views)
Permalink
Re: Moving forward with Lua [In reply to]

On 01/26/2012 08:44 PM, Rob Lanphier wrote:

> http://www.mediawiki.org/wiki/Lua_scripting

Coverage:
https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2012-01-30/Technology_report

http://developers.slashdot.org/story/12/02/01/1429229/wikipedia-chooses-lua-as-its-new-template-language

http://www.reddit.com/r/programming/comments/p4sve/wikipedia_chooses_lua_as_its_new_templatemacro/
--
Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tstarling at wikimedia

Feb 1, 2012, 3:34 PM

Post #5 of 9 (4622 views)
Permalink
Re: Moving forward with Lua [In reply to]

On 29/01/12 09:51, Platonides wrote:
> Doesn't seem to have arrived into the list. Resending.
>
> On 27/01/12 16:17, Platonides wrote:
>> 1) Is there a simple Lua syntax guide?
>> http://www.lua.org/manual/5.2/ looks more like a grammar. Certainly
>> not suitable for end users.

Yes, the first edition of Programming in Lua is available online:

http://www.lua.org/pil/

The reference manual is under a free license, so I imagine we would
fork it on mediawiki.org, incorporating changes for standard library
functions that are changed or removed, and documenting any
MediaWiki-specific libraries.

>> 2) The decision seems to have been Lua vs JS. What discarded a
>> custom language? (such as WikiScripts)

Mostly performance.

>> 3) How hard is it to translate Lua to PHP?
>> (for templates which then get converted into extensions)

I don't know why you would want to do that, but I imagine it would be
no harder than converting between any other pair of weakly typed
procedural languages.

Server side Lua scripts could potentially execute much faster than
Zend PHP, so we might start wondering how hard it would be to convert
the other way.

>> 4) Where would code live?

In a "scripts" namespace.

>> 5) I think it _would_ be possible to make an extension defined with
>> <wikiscript> to take over the page. (avoiding the need of template
>> wrappers)

I'm not sure what you mean by that.

>> 6) What's the expected way of implementing 'libraries' in the wiki?

I think it would be best to follow the Lua conventions here:

http://www.lua.org/pil/15.html
http://www.lua.org/manual/5.2/manual.html#6.3

Basically, a Lua package is a script which returns a table of
functions. There are a number ways to do this, such as:

local P;

function P.foo()
...
end

return P;


A require() function would return the table. Initially require() would
only fetch packages from the local wiki, but when interwiki
transclusion is introduced, it would be expanded to allow libraries on
remote wikis to be fetched. We would isolate the global namespace of
packages to prevent global variable exports. So package invocation
would be along the lines of:

package = require("The package title");
package.foo()


Assuming I understood him correctly, Trevor expressed a preference for
having a function that returns a function value which returns a table
when called, instead of just returning the table directly. For example:

packageFunction = getScript("The package title")
package = packageFunction()
package.foo()

We could have that interface as well, if there's an application for
it. For consistency with the Lua reference manual, I don't think this
interface should be called require().

-- Tim Starling

_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


z at mzmcbride

Feb 1, 2012, 3:51 PM

Post #6 of 9 (4625 views)
Permalink
Re: Moving forward with Lua [In reply to]

Tim Starling wrote:
> On 29/01/12 09:51, Platonides wrote:
>> Doesn't seem to have arrived into the list. Resending.
>>
>> On 27/01/12 16:17, Platonides wrote:
>>> 1) Is there a simple Lua syntax guide?
>>> http://www.lua.org/manual/5.2/ looks more like a grammar. Certainly
>>> not suitable for end users.
>
> Yes, the first edition of Programming in Lua is available online:
>
> http://www.lua.org/pil/

Thanks for this. I've added this link to (the brand new)
<https://www.mediawiki.org/wiki/Lua/Resources>.

I also un-redirected <https://www.mediawiki.org/wiki/Lua> and fleshed it out
a bit.

MZMcBride



_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


owen at wikia-inc

Feb 28, 2012, 2:01 PM

Post #7 of 9 (4216 views)
Permalink
Re: Moving forward with Lua [In reply to]

Keeping this lua thread alive with some news. :)

For anyone interested in playing around with Lua on a wiki, I've enabled the Lua extension on http://lua.wikia.com
It's just a test wiki right now with no real content. It will probably remain a test wiki, although if users want to flesh it out into a more authoritative site, that would be cool too. I am using http://www.mediawiki.org/wiki/Extension:Lua

Right now I've experimented a little bit and run across some questions.

Question 1) is that the right extension to be using? Is there another extension being developed?

Question 2) How to load external Lua code and make it available?

I created an example of an info box generated from lua here: http://lua.wikia.com/wiki/InfoboxLua
However, it's clear that the first thing you need is a template library. I can't figure out how to load a third party lua library and make it available on a page. This probably needs some kind of support in the extension itself. For example:

http://lua.wikia.com/wiki/TestExpand?action=raw

Here I am using the lua loadstring() function to load/eval a simple template library and attempting to execute it in the same page. That doesn't work, although the same code works just fine from the command line.

I originally tried with a template like loadstring('{{Expand.Lua}}' and the template actually expands properly but runs into the same problem. The function created by load string is not actually executable. I'm not sure if it's a sandboxing issue or something else yet, I haven't had time to investigate. There's also a loadfile() function but that reads off the local file system. I suppose I could add things to the local lua include path and load them that way, but it seems better to have a self contained 100% wiki based solution there.

Has anybody gotten that far, or farther with Lua? I'm happy to hack on the extension a little bit if that's the proper solution here.

Owen

On Feb 1, 2012, at 3:51 PM, MZMcBride wrote:

> Tim Starling wrote:
>> On 29/01/12 09:51, Platonides wrote:
>>> Doesn't seem to have arrived into the list. Resending.
>>>
>>> On 27/01/12 16:17, Platonides wrote:
>>>> 1) Is there a simple Lua syntax guide?
>>>> http://www.lua.org/manual/5.2/ looks more like a grammar. Certainly
>>>> not suitable for end users.
>>
>> Yes, the first edition of Programming in Lua is available online:
>>
>> http://www.lua.org/pil/
>
> Thanks for this. I've added this link to (the brand new)
> <https://www.mediawiki.org/wiki/Lua/Resources>.
>
> I also un-redirected <https://www.mediawiki.org/wiki/Lua> and fleshed it out
> a bit.
>
> MZMcBride
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l [at] lists
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


tstarling at wikimedia

Feb 28, 2012, 4:10 PM

Post #8 of 9 (4255 views)
Permalink
Re: Moving forward with Lua [In reply to]

On 29/02/12 09:01, Owen Davis wrote:
> Keeping this lua thread alive with some news. :)
>
> For anyone interested in playing around with Lua on a wiki, I've
> enabled the Lua extension on http://lua.wikia.com It's just a test
> wiki right now with no real content. It will probably remain a
> test wiki, although if users want to flesh it out into a more
> authoritative site, that would be cool too. I am using
> http://www.mediawiki.org/wiki/Extension:Lua
>
> Right now I've experimented a little bit and run across some
> questions.
>
> Question 1) is that the right extension to be using? Is there
> another extension being developed?

Victor Vasiliev has been working on an extension called Scripting
which has a feature set closer to the plans we've been discussing at
WMF. I'm not sure what its level of maturity is. It uses my luasandbox
extension (/trunk/php/luasandbox) rather than shelling out.

> Question 2) How to load external Lua code and make it available?

Victor's code has a function called mw.importModule which may be of
some use.

>
> I created an example of an info box generated from lua here:
> http://lua.wikia.com/wiki/InfoboxLua However, it's clear that the
> first thing you need is a template library. I can't figure out
> how to load a third party lua library and make it available on a
> page. This probably needs some kind of support in the extension
> itself. For example:
>
> http://lua.wikia.com/wiki/TestExpand?action=raw
>
> Here I am using the lua loadstring() function to load/eval a
> simple template library and attempting to execute it in the same
> page. That doesn't work, although the same code works just fine
> from the command line.

I don't think we will support loadstring(), instead all code will have
to be loaded via the module system.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


jamesmikedupont at googlemail

Feb 28, 2012, 10:08 PM

Post #9 of 9 (4212 views)
Permalink
Re: Moving forward with Lua [In reply to]

very cool.

how can we see the lua errors?
how can we load wiki pages as data in lua?

thanks,
mike

On Tue, Feb 28, 2012 at 11:01 PM, Owen Davis <owen [at] wikia-inc> wrote:

> For anyone interested in playing around with Lua on a wiki, I've enabled
> the Lua extension on http://lua.wikia.com
> It's just a test wiki right now with no real content. It will probably
> remain a test wiki, although if users want to flesh it out into a more
> authoritative site, that would be cool too. I am using
> http://www.mediawiki.org/wiki/Extension:Lua
>



--
James Michael DuPont
Member of Free Libre Open Source Software Kosova http://flossk.org
_______________________________________________
Wikitech-l mailing list
Wikitech-l [at] lists
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wikipedia wikitech RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.