-
Notifications
You must be signed in to change notification settings - Fork 168
A Developer's Tour Through Shoes
This is not everything you need to know as a C developer or a Ruby developer that wants to learn the C inside Shoes. It's only a modest attempt to guide you both through the source code that is Shoes.
First I show how the source code tree is set up. Then I attempt to guide you through how Shoes starts up (without sucking on a fire hose of C headers and Macros). But eventually, you'll have to deal with them. Then I talk a little bit about some features the really make Shoes extra cool. Stuff like packaging and the built in manual and the details of the rakefile. If your eyes don't glaze over and fall out, there is a section about how to release a new version of shoes.
Reading this won't make you a clued in developer. At best, I hope to show you how and where the pieces parts fit together so you can explore Shoes with an 'I'm not lost yet" composure. Sadly, you might become lost. There is a lot going on under the hood of Shoes but maybe this guide will be the bread crumbs that will get you back from the twisty passages.
First we need to learn a little bit about the file and directory tree that is Shoes. Lets look in the top level directory (my example is Linux and I'm cd'd into the top level directory, so its relative paths from here on). dist/ is where the new shoes is compiled into, installed or run from. CHANGELOG, COPYING, README and bugs/ are out of date. There are a bunch of files for getting shoes to compile or build or distribute another version of Shoes, files like the Rakefile and rakefile*, use-dep, make, app.yaml.
samples/ is just that -- samples of Shoes code (useful for testing) /fonts we don't care about at this point. static/, is mostly images although static/stubs/ is a concern for developers, as is platform/. Those get used in building/packaging Shoes.
When all the above is pushed aside, we have something that kind of looks like a Ruby gem project and a C project. Because that's what it is. It's neither Ruby nor C, it's both! Two in one!
Shoes is a C application that embeds or contains its own Ruby interpreter. It is not textbook Ruby although it resembles it.
There are built in gems (binary extensions that act like gems) in req/
| Subdir | Purpose |
|---|---|
| bin/ | The shoes main.c lives here |
| lib/ | Where the Ruby code of Shoes lives |
| req/ | The code for Shoes built-in Gems |
| shoes/ | The C code that is the Shoes application |
| Subdir | Purpose |
|---|---|
| binject | Used to package scripts and Shoes together for one click install |
| [bloopsaphone](http://github.com/mental/bloopsaphone) | Ruby binding for writing chiptune-style songs |
| chipmunk | Ruby binding for [Chipmunk Physics](http://github.com/ashbb/shoes_hack_note/blob/master/md/hack029.md) |
| ftsearch | Used by _Try search_ in the built-in Shoes Manual ([a tiny note](http://github.com/ashbb/shoes_hack_note/blob/master/md/hack010.md)) |
| [hpricot](http://github.com/hpricot/hpricot) | An XML parser |
| json | [JSON](http://flori.github.com/json/) implementation for Ruby. Not used in Shoes so far, but just included. |
| sqlite3 | A simple SQL database, if SQL can be called simple |
These are binary gems (C code) that are in every Shoes. They are built like normal native gems with extconf.rb and Makefiles. If you go into the req/ you need to know it is all about gem building in there. Binary Gem conventions apply.
So whats in lib/ ?
| Contents | Purpose |
|---|---|
| shoes.rb | What the C code runs in _its_ Ruby interpreter. The startup up screen of Shoes. |
| shoes/ | The Ruby code to connect to the built in gems in ext/ and to expand some Shoes features (*1) |
*1: ex. class Widget, built-in manual, Shoes.url, complete list of styles, Shoes::LogWindow, cache download files, etc.
Everything above is more or less Ruby normal practices (if you build binary gems). That leaves us with the shoes/shoes/ directory (we covered lib/shoes/ above) There we find the C code that makes Shoes an application on Windows, OSX and Linux. The heart and brain of Shoes and perhaps the soul. It's all .c and .h So we have change our mindset and be C programmers.
For every.c file there is likely to be an every.h file. The .h declares the functions and constants that will be exposed by the every.c file. Usually every.c includes it's matching every.h along with other .h files that point to other native functions that it will be called. It gets stitched up when the object code is linked into a executable.
So what is in shoes/shoes/? Please remember that the Shoes C code has cross platform Macros and #includes. It is not OO or C++. C macros are textual substitutions with the barest support for macro arguments possible. They depend on C level environment variables and the selective inclusion of other *.h files.
| Files | Functionality |
|---|---|
| app.c, app.h | Code to manage the Shoes windows and events as defined by Shoes |
| appwin32.h,appwin32.rc | Windows requirements |
| canvas.c,canvas.h | Cairo and drawing |
| code.h | Defines some Shoes Error codes |
| config.h | Macros that Shoes C uses. Pay attention |
| effects.c,effects.h | Cairo drawing effects |
| HTTP/, http.h | Platform Specific downloading |
| image.c | Load images into Cairo |
| internal.c,internal.h | Some stuff that some C compilers don't provide |
| native/,native.h | See below |
| ruby.c,ruby.h | Where Shoes creates its version of Ruby |
| version.h | Created for you, somehow |
| world.c,world.h | Cross platform helpers |
Basically the above sets up a cross platform abstraction/contract in C with functions or Macros that the native/ files have to fulfill. There are Ruby macros and embedding calls and Shoes macros and for each platform there are other macros and functions for each platform that have to be realized in the C code that native/ C code.
Complicated? Yes! A fire hose of macros and subdirectories in psuedo OO? Yes! If you want to add a widget or method to Shoes or fix a widget in Shoes, you need to know ruby.h, ruby.c and all three platforms in native/
If we learn what happens when shoes initializes then we will learn a lot about Shoes internals (not everything but enough to explore with knowledge later)
All C programs start with the main() function which is in bin/main.c - Actually Microsoft programs start WinMain() but that is the main.c file too. Basically there are some init calls to get win32 or osx setup and then main builds a string of Ruby code that will be evaluated later, and it adjusts some of the argv around. And calls
RUBY_INIT_STACK
ruby_init();
rb_eval_string(bootup);
return ruby_run_node(ruby_options(argc, argv));
RUBY_INIT_STACK is a C macro. Odds are it gets some stack space for the Ruby. ruby_init() which is in shoes/shoes/world.c. Odds are high it sets up the ruby interpreter. rb_eval_string is a call to the ruby interpreter so that string might be useful to understand because it is setting up the ruby level paths. Then it call ruby_options with the argc, argv and pass that into a call to ruby_run_node.
ruby_run_node and ruby_options are C functions (apis) of ruby interpreter.
- ruby_run_node runs ruby interpreter (used to ruby_run in ruby 1.8)
- ruby_options parses command line options for ruby interpreter
Lets scan world.h and world.c. world.h defines a new C type, shoes_world_t which happens to be structure with pointers to interesting things and defines some function prototypes (in world.c). It's important to know that some of the code in world.c is not used or only used in odd circumstances. Basically shoes_init() is all about setting up the shoes_world_t and getting ruby prepped to become Shoes. Lots of interesting things here are worth exploring, but for this tutorial we want look at
shoes_native_init();
rb_const_set(cShoes, rb_intern("FONTS"), shoes_font_list());
at the end of shoes_init(). The later line sets up a Ruby constant "FONTS" in the cShoes (must be or name/ptr to all the embedded ruby context being initialized). The constant is filled from shoes_font_list() which is platform specific. I'll bet native.h and the files in native/ are going to be interesting.
Indeed. native.h defines the interface that any GUI toolkit (windows, mac, linux, something-new) must implement along with the shoes_native_init and shoes_font_list and many more.
I danced past that line in shoes/shoes/world.c that calls shoes_ruby_init.
We are in a twisty passage of inits(), all different, unless they aren't used.
Look at shoes/shoes/ruby.h. Know thee, that this not the same ruby.h that the Ruby embedding API defines, which is also included in at the top to confuse thee. shoes/ruby.h is full of Macros. Of course it prototypes the shoes_ruby_init that is found in shoes/shoes/ruby.c as well as some helper functions that shouldn't have the 'rb_' prefix because most you would think those are in the system ruby.h embedding API or should be in world.c. Welcome to Shoes C, it's not always clean and pretty.
shoes_ruby_init() in shoes/shoes/ruby.c the whole file is where Shoes becomes 'not really Ruby' We arrived at the Heart of Shoes. C functions to manipulate various graphical things and structures abound. Around line 4622 is the the shoes_ruby_init and since we are taking the quick init() tour, lets see what it does.
It sets up Ruby constants and vars. It does some module mixin magic and then it tells the ruby interpreter about every Shoes class, method and arguments to the methods as it sets up a callback or hook back to the C code that should be called in response to a Shoes script calling a Shoes method. Some of that C code to called is in ruby.c Some is into Cairo and Pango and some of it is into those native Shoes files. Serious C Macro goings on in ruby.h, used by ruby.c. It's a lot to absorb and if your like me, it's a lot to admire.
Our startup init() tour ends with lib/shoes.rb where the command line args (if any) can bring up the the manual or the packager or the gem handler or run a script or, if all of the above is missing, brings up the Shoes screen. That's after all of the above setup.
This has been your quick tour of Shoes (in C)
There is one more part of the startup puzzle: The lib/shoes.rb (with support from the C code) does something you might not expect.
This is the Minimal Shoes ( vs. lib/shoes.rb) instead of the Shoes with the fancy startup screen and menus. No, you do not want to replace lib/shoes.rb with with the following, this is for illustration purposes.
# shoes.rb
class Shoes
def self.run path
[nil]
end
def self.args!
Shoes.app{ para 'Hello world!' }
end
end
You need to think about what's going on in there and the Rubinations that make that different from opening the Shoes class and what looks like a monkey patch override of the run and args methods. Took me a while to see that '!' in the def self.args! I'll bet the C code to handle args! is worthy of some understanding
There are other tricks Shoes does to be not-the-Ruby-you-know. Please see Rules of Shoes. It's important info if you want to understand how then Ruby and C code work.
Developers and users who use the command line can do every thing that the Shoes GUI does (shoes/Shoes.rb) and a lot more. When you run Shoes, you get three mutually exclusive choice -- Open the manual, Open (run) a script or bring up the packager. These are the command line equivalents.
If you compiled shoes but didn't install it as the system wide Shoes then the compiled version is in dist/
$ dist/shoes -h
Usage: shoes [options] (app.rb or app.shy)
-m, --manual Open the built-in manual.
-p, --package Package a Shoes app for Windows, OS X and Linux.
-g, --gem Passes commands to RubyGems.
--manual-html DIRECTORY Saves the manual to a directory as HTML.
--install MODE SRC DEST Installs a file.
--nolayered No WS_EX_LAYERED style option.
-v, --version Display the version info.
-h, --help Show this message
You can render the manual to html in a directory of your choosing. I have no idea what --install really does for every OS. It's not something to play with unless you know more. -g allows you to install or manipulate the gems in the Shoes. This can be very useful. Perhaps there is a binary gem that is not cross plaftorm and you don't care if your script runs everywhere. Maybe Shoes.setup doesn't work for you or the error messages vanished when attempting to build a binary gem for your platform.
(need link to what shoes.setup does)
If you have multiple versions of Ruby installed be very careful. That Shoes you compiled into dist/ is a different Ruby which may or may not put it's gems in the same place as another version of Shoes compiled with a different version of Ruby. But for example only, you want the iTunes client gem
$ dist/shoes -g install daapclient
(it fails to compile for me, but back the old days it did. It's only an example. I also have a decent error message to pursue by install it with -g. If it did work, the Shoes script would
require 'rubygems'
require 'net/daap'
The idea behind Packager is that a Shoes script writer should be able to bundle up his Shoes script and all the files and directories it requires into one file (think zip.files) and pass it around. Shoes calls that a .shy file. It works anywhere Shoes is already installed.
Shoes also goes one step farther. You can bundle up your script and directory and Shoes and Ruby code together so the user doesn't even have install Shoes if they have a Windows or Mac (might work in Linux). But wait there's more. You can bundle or package your Shoes app so that it downloads Shoes and Ruby when/if the user needs it. That's the promise and it really did work, once. Some of it still does.
The Ruby code lives in lib/shoes/ in pack.rb and shy*.rb which calls the built-in gem 'pack' and which is 'C' code in req/binject/ext/binject_c. The C code is an psuedo gem. It's build in to Shoes and nowhere else. A .shy file is really just minimalist tar ball build with the lib/shoes/minitar.rb 'binject.c' is called to create more complex installers for windows, mac, and linux, but conceptually it creates a tar ball/zip to hold the script and the installer code and if asked, all the Ruby and Shoes code and libraries to run the Shoes script. All packageup in an exe or dmg that can be double click like the installers we are used to. One could even tell the packager to create an much smaller exe or dmg that downloaded the Shoes and ruby libraries from the network. I could package up myscript on my Linux box and Windows or Apples users could just double click the download icon/package. A Windows user could do the same thing. An OSX user could do the same thing. Freaking cool if it wasn't in disrepair.
How does that work [used to work?] Think about the what user of the download needs: the double click on the icon should start up an installer that gets Shoes and Shoes' version of Ruby from inside the download file or it should downloud what it needs from the shoes website.
There are two parts to packaging. Creating the installer with or w/o net_install and what the user runs. lib/shoes/Pack.rb (with calls to binject) creates the installer (exe, dmg) that includes shoes/ruby or includes just enough to download shoes.
In shoes/static/stubs are the plaftorm specific files that are involved. These are Windows.exe or OSX.dmg (+plus shell script) that are supposed to know if they have a ruby/shoes bundle attached and if they don't, then download them and then run the script. Seriously, that used to work.
At the time I write this, the shoes website doesn't offer an OSX to be included in the package if that was the packers choice and the script/dmg that is run doesn't know how to download the missing pieces from the net. Windows net install only works if you run Windows (ick) but you can bundle up Shoes/Ruby/ into a 7MB download. Half done for Windows. 100% fail for OSX
Linux is a special case. When all is said and done, The Linux packing (into a.run) only knows about Net_install and it's only going to work if the user happens to be running Ubuntu 32 bit Linux. But if they do, it works. There's no point in creating a 7MB linux download that includes all the Shoes and Ruby code for every version of Linux out there. Also, in the pesky reality, lib/shoes/minitar.rb can't handle soft links properly and Linux lives on soft links pointing to libraries and this and that, here and there.
Any new gems installed by Shoes' are in ~/.shoes/ (as are many other things). This is a blessing and huge gotcha. A blessing for pure Ruby gems and a endless source of hellfire for binary gems (AKA native gems). Pure Ruby gems work as intended and they don't touch your system's Ruby Gems (which would require admin privileges) . Unless you happen to have versions of gems in ~/.shoes/compiled for or by different ruby's (1.8.7 vs 1.9.1) and that can bite big time on binary or native gems.
Let's examine a recent issue from the mailing list. The gem 'serialport' is native. Long ago and far away, I installed that in ~/.shoes - version 1.0.1, compiled against Ruby 1.8.7 headers and shared libs. Now serialport is 1.0.4 and I'm running Shoes policeman which requires Ruby 1.9.1 headers and shared libraries for compiling and linking. No problem right? No, several problems can nail you. Assuming you can even install it. I can't. Lets see why and in the process learn about Shoes and gems and binary compiling.
First off, you can't compile native/binary gems from inside Shoes unless you and your target users have the correct developer tools installed and you and they use the proper versions of the toolchain. If you're running Windows, you need the right version of MingW installed and your users will need that version. Same problem for OSX and Linux and just for fun the gem versions had better match.
In my case the system thinks Ruby is 1.8.7 and gem is 1.3.5, although the Policeman version of Shoe I want to install with is Ruby 1.9.1, Gem 1.3.7., if at any time, Shoes attempts to use the the 'systems' version of gems or the 'systems' version of headers or shared libraries its going fail to compile or even worse compile and fark up when loaded or when called.
The Shoes you download does not included every possible shared library that a binary-native gem might want to compile and link against so it has to go outside the Shoes Sandbox to find the shared libraries and headers in the system . If your installed version of Ruby and Gems and tool chain is exactly the same as the versions used to build the shoes you downloaded and then attempt to install the native gem, and you have the needed build environment. It might work. That's how I got serialport 1.0.1 installed in `/.shoes/ way back when 1.8.7 was the only Ruby to encounter. Pure dumb luck.
Perhaps that explains the extraordinary effort Shoes makes to compile its own 'psuedo' gems for sqlite3, hpricot and other native gems. It also explains why video widget support is in a world of hurt. VLC has to link outside the Shoes sandbox since its not a built-in gem.
Shoes built-in manual is written in English with a translation to Japanese. The text lives in static/manual-en.txt or manual-ja.txt along with the .png images. The text files use a markup language (?which one?) that is converted to html/css (see static/manual.css) and displayed with the help of the code_highlighter*.js files in some situations. I haven't looked into this magic very deeply.
You can select by setting the environment variable like this:
SHOES_LANG = en_EN.UTF-8 or SHOES_LANG = ja_JP.UTF-8
To be written
To be written
To be written
To be written