-
Notifications
You must be signed in to change notification settings - Fork 0
Lmod: An Environment Module System based on Lua, Reads TCL Modules, Supports a Software Hierarchy
License
dajon/Lmod
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Lmod 6.1: Bug Fixes: a) The command module spider would fail to find a module if a site had the spider cache file dbT.lua files and a user had personal modulefiles. This is now fixed. b) Two or more .version files could confused Lmod. This is now fixed. c) Lmod can now find UPPER CASE string when doing "module key ..." d) Lmod now ignore the value of PAGER, instead it uses LMOD_PAGER (default "less") and LMOD_PAGER_OPTS (default "-XqRMEF"). This allows for consistent behavior under systems like the Cray and Mac OS X. Lmod 6.0.1: Bug Fixes: a) This version now contains the contrib/tracking_module_usage directory. b) This version fixes a bug when trying to run module avail or module spider with an old cache file. Lmod Version 6.0 Features: a) Full support for global RC files as well as .modulerc files. This means that $MODULERCFILE will be read along with ~/.modulerc. Also .modulerc files are read when they appear in the same directory and the version files (i.e. the same directory as 1.2 or 2.1.lua). One caveat, Setting the default module version in $MODULERCFILE or ~/.modulerc is not supported due Spider cache issues b) Instructions on how to save module usage data to a MySQL database along with scripts to analyze the usage are in contrib/tracking_module_usage/README. c) Adding an exit hook. This makes for more accurate tracking of module usage. See README above as to why this is a good idea. d) Some minor speed-ups and minimizing of directory tree walking. Lmod Version 5.9.3 Features: a) Lmod has been ported to Lua 5.3. b) Thanks to Kenneth Hoste, a parameterized script, called update_lmod_system_cache_files, to build system spider caches has been added as part of Lmod. There is no need to modify createSystemCacheFile.sh (in contrib/BuildSystemCacheFile) to suit your site. c) Sites and users now have a choice about restores from a saved collection. Normally Lmod will follow defaults when restoring. That is if you saved a collection with a default module, Lmod will load the default upon restore, even if the default module has changed. By configuring Lmod, you can keep the same versions. Users can control this by setting LMOD_PIN_VERSIONS to true, or --pin_versions. Lmod Version 5.9.1 Bug fixes a) The spider cache is now ignored on unloads, show as well as loads. b) Lmod treats symlink directories and files much more quickly. It also means that Lmod DOES NOT resolve two symlink directories pointing to the same place. It is too expensive to resolve that on parallel file systems. c) Initialization with Csh now works even when $TERM is not set. Features: a) Bash and Zsh tab-completion now support module restore <tab> with the savelist. b) The standard setup sets LMOD_VERSION as part of the startup procedure. Lmod Version 5.9 Feature: a) Several changes to support faster avail with the spider cache. This change requires updating the way the system spider cache is built. See contrib/BuildSystemCacheFile/README.txt. On an ssd laptop "ml av" is twice as fast. On two different HPC system I have seen 4 times speed improvement. Bug Fixes: a) Fixed a bug where LUA_INCLUDE was not set. Lmod Version 5.8.6 Bug Fixes: a) Fixed a bug several modulefiles where marked as default b) Two different fixes which will improve the speed of reading the the cache file. c) Configure will now abort if either tclsh or lua.h is required but not available. Lmod Version 5.8.5 Features/Bug fixes: a) Fixed bug where the system spider cache was ignored. This bug was introduced in 5.8. It is fixed now. b) Support for disabling same name autoswapping added. This is not active by default. c) Removed extra newlines when doing module list. d) Correctly handle symlink directories with MODULEPATH. e) Improved Spider support where module names have "-" f) Fixed problem where lmodrc.lua was left out in tar ball. g) Fixed problem where src/ignore_dirs_converter was left out in tar ball. Lmod Version 5.8 Features/Bug fixes: a) Lmod support auto swapping instead of producing an error. b) The init/bash.in and init/cshrc add a trailing colon to MANPATH if it is empty c) Fixes bug in ml tab completion for zsh. d) setenv, prepend_path, etc now trim any leading or trailing blanks around name: prepend_path(" MANPATH ","/opt/apps/...") is now the same as prepend_path("MANPATH","/opt/apps/...") e) The module command (and not ml) support combining of single letter options. "module -dw=60 avail" is the same as "module -d -w=60 avail". f) If a module collection file gets corrupted, Lmod now reports that the collection file is corrupted and tells the user to remove it. g) Lmod now support "module swap <name>" which is exactly the same as "module load <name>". The module <name> is unloaded and then reloaded. h) Sites can configure for exporting the module shell function in bash or not. Lmod Version 5.7.5: Features/Bug fixes: a) "module use" and "ml use" now will work with tab completion under bash. (It worked before for zsh). b) New option, --spider_timeout sec is added. Reason: When Lmod can not load a module, it does a module spider to see if the module exists but isn't loadable. Now Lmod doesn't do the spider when in LMOD_EXPERT mode. It also times out after 2 seconds. If Lmod can't tell you quickly if a module exists, why do it. c) New configuration option --with-ignoreDirs. By default Lmod will ignore directories named .svn, .git, .hg and .bzr in the Module directories. If you use CVS, RCS or SCCS to source control your modulefiles, you will need configure Lmod to know about those directories. d) New avail hook added. You can regroup the output of avail. The avail hook gets a table of current directories that avail will list. Your routine can relabel them. The input looks like this: t = { ['/path/to/IntelCmp/mfiles'] = "/path/to/IntelCmp/mfiles" ['/path/to/core/mfiles'] = "/path/to/core/mfiles" } Your routine can relabel and change the labels t = { ['/path/to/IntelCmp/mfiles'] = "Compiler Dependent" ['/path/to/core/mfiles'] = "Core Modules" } You can also group them t = { ['/path/to/IntelCmp/mfiles'] = "Modules" ['/path/to/core/mfiles'] = "Modules" } In the last case intel modules and core modules will be grouped together. e) Better handling of zsh tab completion files. Lmod will continue to install even when a site's zsh setup is broken. f) New option, --show_hidden, has been added. This is used with avail to show "hidden" modulefiles. That is module files that start with a "." Lmod Version 5.7.4 Features/Bug fixes: a) Improved support for redirecting output to stdout. Now show, and help will write to stdout if LMOD_REDIRECT=yes. b) module show unknown will set the return error code if "unknown" doesn't exist. c) "module --terse spider phdf5" will now produce a terse output. d) a module file can do setenv("FOO","") and set the env.var to an empty string. e) Support for empty strings "" in paths are better supported. Lmod Version 5.7.2 Features: a) This version supports (an optional) sending the output of avail, list, spider and keyword stdout instead of stderr. This available as a configure option, an environment variable (LMOD_REDIRECT=yes) or module --redirect avail. Note that if you use this option the internal pager is off. Lmod Version 5.7 Features: a) Lmod is now more efficient when loading and unloading modulefiles that modify MODULEPATH when there is no cache file. b) Lmod now reports that lua-term is active or not when "module --config". This is there so that if Lmod is built on a system that as a system lua-term on one and is installed on a different system that doesn't. c) module spider now reports module version in "parseVersion" order. (i.e 0.7.10 is newer than 0.7.2) d) Internally Lmod sets LC_ALL=C so that a user's Locale does not effect its behavior. Bug Fixes: a) Lmod can now handle TCL modulefiles that use "puts --nonewline ..." b) TCL whatis with multiple arguments is now correctly handled. c) Changed require("posix") to local posix=require("posix") Lmod Version 5.6.3 Bug Fixes: a) Lmod now correctly handles prepending multiple directories to MODULEPATH in one prepend_path. Lmod Version 5.6.2 Bug Fixes: a) LMOD_EXPERT is now also turns on quiet mode. It was removed accidently in 5.6.1 and is now restored. b) Lmod had a bug where it tracked prereq, conflict and family commands in module collections where it didn't need to. This means that some users will have to rebuild their module collections when they shouldn't have to. Lmod Version 5.6.1 Features: a) The environment variable LMOD_OPTIONS can be use to specify command line options to LMOD: export LMOD_OPTIONS="--quiet" b) The environment variable LMOD_PAGER can use to specify the pager that Lmod will use, otherwise PAGER is used or "more" c) the -q or --quiet option suppresses messages and warnings (but not errors). Bug Fix: a) Executing "module list" when there no modules loaded is a message and not a warning. Lmod Version 5.6 Major Bug Fix: Two bugs I have un-earthed cancelled each other all related to saved collections. The upshot is that many users will have get a message that they have to rebuild their collection because the modules have changed. This isn't true but they will have to anyway. I now have tests which will guard against this from happening again. I apologize for this bug. Bug Fixes: a) Lmod is supposed to error out if the modules in a named collection changed in a way such that the loaded modules would be different from loading directly. This is fixed. b) This version correctly handles prepending to the MODULEPATH outside of Lmod c) Lmod now correctly handles symlinks to a template in the same directory. Lmod also support the three ways to mark a default with this setup as well. d) Lmod now correctly handles "module-info mode" command in tcl modulefiles. e) tcl2lua.sh improvements to supports Cray modules. Feature: a) Support for .modulerc in modulefile directory only. This file can be used to specify the default module only. No support for System .modulerc or user ~/.modulerc Lmod Version 5.5.1: Bug Fixes: a) Fixed bug where setting the priority twice caused an error. This is now fixed. b) A meta module that starts with "." will not seen by avail and spider. Lmod Version 5.5: Feature: a) priorities can now be used with "module use": $ module use --priority 100 /path/to/modulefiles Bug Fixes: a) Configuring on a system where tclsh is not found is handled better. b) When doing module avail where MODULEPATH points only to non-existant path it now reports that Lmod can not do a module avail. Lmod Version 5.4.2: Feature: a) Sites with common home file system can use named collections that are differentiated by LMOD_SYSTEM_NAME. This means that if the environment variable LMOD_SYSTEM_NAME is set to "foo" then the default collection will be saved to and restored from "default.foo". Other named collections work the same way. Bug Fix: a) TCL modules can now have: puts "message" as well. Lmod Version 5.4.1: Added License file to tar ball. Lmod Version 5.4: Features: a) prepend_path now supports a priority. This means that no matter when a module is loaded a path will pushed to be first: prepend_path{"PATH","/first",priority=100} -- lua prepend-path PATH /first 100 # tcl Note the use of {} braces for lua. The braces are only required when specifying the priority, otherwise parens work fine. Priority works for append_path as well. It just pushes paths to the right end. By default, a path has a priority of zero. Paths will a priority of 100 are grouped together. Paths with a priority of 1000 are grouped together to the left of the paths with a priority of 100. The priority for prepend_path are useful for wrappers. The priority for append_path is great for catch-alls. b) Lmod now generates "LOADEDMODULES" and "_LMFILES_" to match TCL/C env. modules. c) Lmod now purges modules in reverse order that the module were loaded. This helps when reading Cray modulefiles. d) Lmod ships with lua-term and JSON4lua but will use the system lua-term and lua-json if available. e) added "--raw" to "module --raw show module". This just prints out the "raw" modulefile. f) Lmod now adds the env. var LMOD_SYSTEM_NAME to the user cache file. So if LMOD_SYSTEM_NAME is Bar and on a intel system the cache file will be ~/.lmod.d/.cache/moduleT.Bar_x86_64.lua g) To match TCL/C modules, the reverse of the module file commands remove_path, unload, etc now does nothing rather-than producing an error. h) Support for a .version file with time to change the default: #%Module1.0################################################## set ModulesVersion "1.4.3" set NewModulesVersion "1.6.5" set NewModulesVersionDate "2014/04/25" Note that the date is in yyyy/mm/dd format. This way it is neither an American or European format. i) To support both rpm and debian package rules, the system .settarg.lua file is renamed to settarg_rc.lua. The .lmodrc.lua file has been renamed to lmodrc.lua. User versions of these files remains the same: ".settarg.lua" and ".lmodrc.lua". Bug Fixes: a) There was a bug with avail and spider where it would not find modules with a dash. This was fixed between version 5.2.3 and now. b) There was a bug with "module -r spider "^phdf5/1.8.12$" where it would say that the module wasn't there and yet it was. This has been fixed. c) Improved the wording of "module load foo/1.2" when foo/1.2 existed and when it didn't. (Thanks to Alex Moskalenko for the fix) d) Lmod would do wrong when "module unload gcc mpich". It would make mpich inactive. This is fixed. e) Fixed bug where if there is a symbolic link to the modulefile it wouldn't mark the default module correctly. Lmod Version 5.3.2: Bug Fix: a) Fixed problem with symlink directories and picking the right default. b) Fixed a bug where .version did not find the right default. Lmod Version 5.3.1: Feature: a) added tool "sh_to_modulefile" to convert a bash shell script into a lua based shell script.: $ sh_to_modulefile -o intel-14.0.2.lua /a/intel/bin/iccvars.sh intel64 note that you can give argument to the shell script. Lmod Version 5.3: Features: a) Searching for avail, spider and list are case-insensitive. Lua regexp searching now requires a "-r" option b) The modulefile function isloaded() takes atleast, between and latest: isloaded(atleast("Foo","1.2")) c) Spider searching improved. When an exact match is found Lmod reports other possible matches: R: R/2.1.5 Other possible modules matches: PrgEnv, greenlet, parmetis, r Bug Fix: a) Quotes inside a whatis or help message are properly escaped. b) Better handling of nested modules by fixing load order. Lmod Version 5.2.4: Feature: a) Lmod can be configured to ignore TCL module files. Lmod Version 5.2.3: Bug Fixes: a) Prereq now works with a version. b) The function "always_load" works with show. c) The "make install" now works and doesn't complain about not being a git repository. Lmod Version 5.2: Features: a) Modulefiles can mix load commands and setenv's and still be saved and restored. b) In Lua modulefiles you can modify load function behavior: load(atleast("a","1.2")) -- load module "a" with version 1.2 -- or higher. load(between("b","1.2","1.4") -- load module "b" with version between -- 1.2 and 1.4. load(latest("c")) -- load latest version of "c", ignore -- marked default. This only works in Lua module files, and works on all load functions: load, always_load. c) In Lua modulefiles you can modify prereq function behavior: prereq(atleast("a","1.2")) -- Module "a" must be version 1.2 -- or higher. prereq(between("b","1.2","1.4") -- Module "b" must be version between -- 1.2 and 1.4. prereq(latest("c")) -- Module "c" must be latest possible This only works in Lua module files, and works on all prereq functions: prereq, prereq_any. Lmod Version 5.1.5: Bug Fixes: a) Changes in the terse output for avail and spider. A directory that contains module versions has a trailing "/" b) The environment that a module sees (and only in that environment). Lmod defines the following variables shown with their current values: LMOD_VERSION: 5.1.5 LMOD_VERSION_MAJOR: 5 LMOD_VERSION_MINOR: 1 LMOD_VERSION_SUBMINOR: 5 c) Module function to do version comparison (only for lua based modulefiles): if (convertToCanonical(LmodVersion()) > convertToCanonical("5.0")) then -- Do something that is only valid for Lmod 5.0 or greater end The function convertToCanonical() knows how to deal with alpha, beta, and rc version and the function LmodVersion() returns the current version of Lmod. Lmod Version 5.1.1: Bug Fixes and internal improvements a) Lmod can now find all the dot module files. b) Fixed bug with inherit() c) Settarg nolonger tries to control all TARG_* variables. d) Better handling of "Rebuild cache ..." line. e) Improvements to the execute function. f) "module --mt" prints the module table g) "settarg --stt" prints the settarg table. Lmod Version 5.1.0: Version 5.1.0 now supports the following new features: a) module --ignore_cache which tells Lmod to ignore the cache even if it exists. b) module --config reports how lmod was configured. c) Lmod no longer walks the directory tree on command like list and unload. d) The pager is now used only when TERM is defined and stderr is connected to a tty (this includes a pty: pseudo tty). e) The system startup scripts (init/profile, and init/cshrc) that are usually linked to modules.sh and modules.csh in /etc/profile.d contains now modulefiles: lmod and settarg. Please consider using this as defined or moving these module files into your normal MODULEPATH. f) Support for settarg has been added for Bash, Zsh and Tcsh users. g) See README.old for features added in earlier versions. ------------------------------------------------------------------------ What is Settarg? ------------------------------------------------------------------------ Settarg is new module that accesses a new tool to dynamically and automatically updates "$TARG" and a host of other environment variables. These new environment variables encapsulate the state of the modules loaded. *** NOTE **** Please note that settarg feature is optional and is implemented as a module. If and only if the settarg module is loaded you get the new behavior. You can load and unload the settarg module at will to turn on and off these new features. Settarg does work for tcsh users as well a feat not to be scoffed at! Unfortunately regular csh is missing necessary features for it to work. *** NOTE **** For example, if I have the gcc/4.7.2 module loaded and the new settarg module loaded I get the following variables defined in my environment. TARG=OBJ/_x86_64_06_1a_gcc-4.7.3 TARG_COMPILER=gcc-4.7.3 TARG_COMPILER_FAMILY=gcc TARG_MACH=x86_64_06_1a If I change my compiler to intel/13.1.0, these variables change to: TARG=OBJ/_x86_64_06_1a_intel-13.1.0 TARG_COMPILER=intel-13.1.0 TARG_COMPILER_FAMILY=intel TARG_MACH=x86_64_06_1a This was designed to help me as a developer of software applications where I am constantly changing compilers, mpi stacks and other modules. These variable are used to set the compiler flags and other things in my Makefiles to control how my application is built. In my case $TARG the directory is where the the objects, libraries and executables are placed during the build process. It easy to have the title bar reflect the dynamic state of your modules. You can abbreviate items mentioned in the titlebar. I have found this combination of settarg and Lmod to be a powerful development tool. Even if this workflow is not yours, you may find parts of this useful. For a more complete tour of settargs features see: contrib/settarg/Settarg.txt and the Make example in contrib/settarg/make_example.
About
Lmod: An Environment Module System based on Lua, Reads TCL Modules, Supports a Software Hierarchy
Resources
License
Stars
Watchers
Forks
Packages 0
No packages published
Languages
- Lua 67.6%
- Roff 12.3%
- TeX 8.5%
- Tcl 6.6%
- Shell 2.7%
- Python 1.9%
- Other 0.4%