A require() function, to load and execute other scripts | Voters

A require() function, to load and execute other scripts

tracked

SuzannaLinn Resident

some ideas on how it could work:

Required scripts are saved as a different type of script: modules, with a new compiler option: compile to "SLua scriopt" or to "Slua module".
Modules are executed when required and return a value, with a
return
statement, for instance a table with functions and constants.
Scripts call modules as
myModule = require("Module name")
, the returned value is assigned to the variable
myModule
, using the script's memory space.
The module must be in the object contents with the script that requires it.
If the module is not in the object, its running state is disabled, or it doesn't return a value,
require()
will return nil.
Once it returns, the module is removed from memory and its variables are garbage-collected.
Modules are executed synchronously, the main script is paused until the module returns.
Modules don't receive events, any events that arrive while the module is executing are queued in the main script.
Modules can't require other modules, only scripts can use
require()
.

March 22, 2025

SophieJeanneLaDouce Resident

I'm not sophisticated enough to say anything intelligent on the difficulties of implementing require(). I'll just throw out my use case and motivation ~ code management. 
For me, all my non-trivial LSL code resides on my computer, checked into git, accessed via Firestorm's #include. I'll argue it's a superior way to manage code than a folder full of copy/paste/edit scripts.
I'm very strongly of the opinion some way to insert code into a certain place in a script (#include) is a "must" or close to it. Otherwise my incentive to use SLua over LSL goes way down, despite all that SLua brings to the table. 
Whether this is done with the name require() or named something else isn't important to me. 
Thank you. :)

Janet Rossini

The FireStorm #include, which in essence just copies a file into the current script at (just before?) compile time would probably provide most of what we need. 
A more clever (and difficult) thing to do would be to have something like link messages ... except without the slow and awkward event-event part, amounting to a direct call to a function in a separate script in inventory. 
In many objects these days, we "get around" the 64K limit by including several scripts and linking around in them. Possibly the 64K limit needs to b relaxed, perhaps at a cost in Land Impact.

animats Resident

This is surprisingly hard to design. It (has given the Roblox Luau people problems.)[https://github.com/luau-lang/luau/pull/1013]. Most of these problems have come up before in dynamic languages. The simple case is simple. The complex cases, most of which lack a strong use case, have overly complex semantics. Lua "require" is basically "eval" of  a file: go read that file and execute it. SL doesn't allow "eval", for good reasons. It's a run-time thing. 
I'd argue for a very limited form of require()
, with roughly the same capabilities and limitations as the #include
 feature of Firestorm's pre-processor. Please keep it simple and get something useable into users' hands.
More than one require
 of the same file should only include it once. Then we can avoid that #ifndef FOO / #define FOO / #include "foo" / #endif
 include guard hack Firestorm borrows from C.
Is it possible to remove dead code when compiling? Firestorm does that, and it means you can have libraries of useful functions without worrying about pulling in too much. That's easier in LSL, because you can't alias a function name and because #include
 is done in a prepass. Trickier to do in Luau, but desirable.

Harold Linden

animats Resident
>I'd argue for a very limited form of require()
, with roughly the same capabilities and limitations as the #include
 feature of Firestorm's pre-processor. Please keep it simple and get something useable into users' hands.
> [...]
> Is it possible to remove dead code when compiling? Firestorm does that, and it means you can have libraries of useful functions without worrying about pulling in too much. That's easier in LSL, because you can't alias a function name and because #include is done in a prepass. Trickier to do in Luau, but desirable.
Absolutely, that's exactly what we're aiming for, along with support for dead code elimination for unused imports. To optimize cross-module effectively with Luau, you really have to pull everything into a single translation unit, giving you something close to #include
-like semantics, but with a separate globals
 table for modules to force encapsulation.
We're not going to overly-complicate things such that it takes ages for people to get access to something useful, there'll be no 5-year multi-stakeholder working groups for this :)
As you mention, the aliasing problem does complicate things (DCE in LSL itself is pretty trivial,) as Luau doesn't  currently do any tree-shaking to get rid of unused function references. We're in a pretty fortunate position as we don't have any existing modules we need to support, so we can impose additional restrictions on modules that Luau proper would not be able to impose to allow effective tree-shaking.
We're looking at a number of options, including forcing modules to only use pure expressions at the top level, as well as forcing explicit declaration of exports that can be resolved statically if you want that tree-shaking / dead code elimination behavior. Ideally we end up with something that's compatible with Luau proper, but allows us to do extra optimizations that are only generally relevant to SL as a compile target.
We expect that those with a background in programming language theory will have thoughts on what makes sense here, so we're going to put together a proper an RFC once we've collected our thoughts and done a review of what others have done in similar spaces. Existing tree-shakers and bundlers for CommonJS are the most obvious sources of inspiration.

Harold Linden

marked this post as

tracked

Harold Linden

We intend to eventually implement require()
 (or something like it,) although there's dragons lurking there, so it will be a bit. In particular, the current script compiler has no concept of object inventory, so resolving a require()
 at compile-time is tricky. Handling the require()
 at runtime would be possible, but because then your script may block on loading the second asset and hurt script performance on reset, not to mention that you then can't tree shake the require()
d script's code at compile-time to remove functions that the script will never use or do function call inlining.
For those of you familiar with the JS world, implementing require()
 as the Luau REPL does would cause problems like JS had for over a decade with CommonJS bundling that was partially solved by switching to ESM. Scripters in SL care a lot about script bytecode size, where most users of Luau do not, and we need to make sure we're not implementing features that you'll end up having to work around to get script memory usage down. Ideally you should not have to use external tools like LSL-PyOptimizer (although I'm thankful they exist) to make your script compile in an efficient form, it's the compiler's job to generate code that isn't bloated and / or slow.
We'll publish a more detailed write-up about our thoughts on what scripters need / want from require()
 along with the implementation considerations sometime soon.

nya Resident

This would dramatically reduce the complexity of scripts that currently have to be thousands of lines long.

Thornotter Resident

Good idea.
Luau already has an export
 keyword designed to be used with require()
, so it would be better to use that official method instead of the return
 method used by other platforms.
Also not sure that modules should be aggressively unloaded for memory management. There are valid uses for modules with persistent internal global state.
A different resource type might not be needed. For example, require
 could be made to only succeed for non-running scripts.