diff --git a/Fuel/Fuel.pillar b/Fuel/Fuel.pillar index 0d400cc..7f212b1 100644 --- a/Fuel/Fuel.pillar +++ b/Fuel/Fuel.pillar @@ -49,8 +49,7 @@ Before going into details we present the ideas behind Fuel and it's main feature ; Object-Oriented Design : A requirement from the onset was to have a good object-oriented design and to avoid special support from the virtual machine. In addition, Fuel has a complete test suite (over 600 unit tests), with a high degree of code coverage. Fuel code is also well-commented, both for classes and methods. -!!! Installation and demo -@Installation +!!! Installation and Demo Fuel 1.9 is available by default in Pharo since version 2.0 of Pharo. Therefore you do not need to install it. The ""default packages"" work out of the box in Pharo 1.1.1, 1.1.2, 1.2, 1.3, 1.4, 2.0, 3.0 and 4.0 and Squeak 4.1, 4.2, 4.3, 4.4, 4.5. @@ -90,7 +89,7 @@ materializedArray third value: 'The materialized block closure can be properly evaluated.'. ]]] -!!! Some links +!!! Some Links - The home page of Fuel is *http://rmod.lille.inria.fr/web/software/Fuel*. - Fuel's source code is stored at *http://smalltalkhub.com/#!/~Pharo/Fuel*. @@ -104,7 +103,7 @@ materializedArray third !! Getting Started @GettingStarted -!!!Basic examples +!!!Basic Examples Fuel offers some class-side messages to ease more common uses of serialization (==serialize:toFileNamed:==) and materialization (==materializeFromFileNamed:==). The next example writes to and reads from a file: @@ -163,7 +162,7 @@ Fuel does not care to what kind of stream it writes its data. This makes it easy gzip close ]. ]]] -!!!Showing a progress bar +!!!Showing a Progress Bar Sometimes it is nice to see progress updates on screen. Use the message ==showProgress== in this case. The progress bar functionality is available from the ==FuelProgressUpdate== package, so load that first: @@ -207,7 +206,7 @@ graph, there is no unique way of serializing it and because of this Fuel offers @@authorToDo JF dynamic and static? which is which? -!!!Default globals +!!!Default Globals By default, Fuel considers the following objects as globals, i.e., it will store just their name: @@ -220,7 +219,7 @@ By default, Fuel considers the following objects as globals, i.e., it will store @@authorToDo JF But what about the inverse of 3.2: DO export a certain global -!!!Duplication of custom globals +!!!Duplication of Custom Globals With this following code snippet, we show that by default a global value is not serialized as a global. In such a case it is duplicated on materialization. @@ -259,7 +258,8 @@ aSerializer [ (FLMaterializer materializeFromFileNamed: 'g.fuel') == SomeGlobal ] assert. ]]] -!!!Changing the environment +!!!Changing the Environment +@sec:ManagingGlobals The default lookup location for globals is ==Smalltalk globals==. This can be changed by using the message ==globalEnvironment:== during serialization and materialization. @@ -338,9 +338,9 @@ User >> fuelAfterMaterialization !!!Substitution on Serialization Sometimes it is useful to serialize something different than the original object, without altering the object itself. Fuel proposes two different ways to do -this: a dynamic way and a static way. +this: dynamically and statically. -!!!!Dynamic way +!!!!Dynamically You can establish a specific substitution for a particular serialization. Let's illustrate with an example, where the graph includes a ==Stream== and you want to serialize ==nil== instead. @@ -364,9 +364,9 @@ objectToSerialize := { 'hello' . '' writeStream}. After executing this code, ==materializedObject== will contain ==#('hello' nil)==, i.e. without the instance of a ==Stream==. -!!!! Static way +!!!! Statically You can also do substitution for each serialization of an object by overriding its ==fuelAccept:== method. Fuel visits each object in the graph by sending this -message to determine how to trace and serialize it. +message to determine how to trace and serialize it. The argument of the message is an instance of a ==FLMapper== subclass. As an example, imagine we want to replace an object directly with nil. In other words, we want to make all objects of a class transient, for example all ==CachedResult== instances. For that, we should implement: @@ -423,7 +423,7 @@ In this case, the substituted user (i.e., the one with the empty history) will b In the same way that we may want to customize object serialization, we may want to customize object materialization. This can be done either by treating an object as a globally obtained reference, or by hooking into instance creation. -!!!! Global reference +!!!! Global References Suppose we have a special instance of ==User== that represents the admin user, and it is a unique instance in the image. In the case that the admin user is referenced in our graph, we want to get that object from a global when the graph is materialized. This can be achieved by modifying the ""serialization"" @@ -443,7 +443,7 @@ User >> fuelAccept: aGeneralMapper During serialization the admin user won't be serialized but instead its global name and selector are stored. Then, at materialization time, Fuel will send the message ==admin== to the class ==User==, and use the returned value as the admin user of the materialized graph. -!!!! Hooking into instance creation +!!!! Hooking Into Instance Creation Fuel provides two hook methods to customise how instances are created: ==fuelNew== and ==fuelNew:==. @@ -540,12 +540,13 @@ the class shape. Now imagine we previously serialized an instance of ==Point== a +Example of changes to a class>file://figures/ClassChanges.png|width=70|label=figClassChanges+ +@@authorToDo Fig reference is broken. -Let's start with the simple cases. If a variable was ""inserted"", its value will be ==nil==. If ""removed"", it is also obvious: the serialized value will be -ignored. ""Change of Order"" of instance variables is handled by Fuel automatically. +Let's start with the simple cases. If a variable was ""inserted"", its value will be ==nil==. If it was ""removed"", it is also obvious: the serialized value will be +ignored. The ""change of Order"" of instance variables is handled by Fuel automatically. A more interesting case is when a variable was ""renamed"". Fuel cannot automatically guess the new name of a variable, so the change will be understood by Fuel -as two independent operations: an insertion and a removal. To resolve this problem, the user can tell the Fuel materializer which are the renamed variables by +as two independent operations: an insertion and a removal. To resolve this problem, the user can tell the Fuel materializer which variables are renamed by using the message ==migratedClassNamed:variables:==. It takes as first argument the name of the class and as second argument a mapping from old names to new names. This is illustrated in the following example: @@ -566,18 +567,19 @@ FLMaterializer newDefault Lastly, Fuel defines the message ==migrateClassNamed:toClass:variables:== that combines both ""class and variable rename"". -Additionally, the method ==globalEnvironment:==, showed in Section *@ManagingGlobals*, is useful for migrations: you can prepare an ad-hoc environment -dictionary with the same keys that were used during serialization, but with the new classes as values. +Additionally, the method ==globalEnvironment:==, shown in Section *@sec:ManagingGlobals*, is useful for migration of global variables: you can prepare an ad-hoc environment dictionary with the same keys that were used during serialization, but with the new classes as values. -@@note A class could also change its ""layout"". For example, Point could change from being ""fixed"" to ""variable"". Layout changes from fixed to variable format are automatically handled by Fuel. Unfortunately, the inverse (variable to fixed) is not supported so far. +@@authorToDo reference is broken? + +@@note A class could also change its ""layout"". For example, Point could change from being ""fixed"" to ""variable"". Layout changes from fixed to variable format are automatically handled by Fuel. Unfortunately, the inverse (variable to fixed) is not supported yet. %=========================================================================% !! Fuel Format Migration -Until now, each Fuel version has its own stream format. Furthermore, each version is ""not"" compatible with the others. This means that when upgrading Fuel, we -will need to convert our serialized streams. +Until now, each Fuel version has used its own stream format, which is ""not"" compatible with the format of other versions. This means that when upgrading Fuel, we +will need to convert our serialized streams. This is done by using the old version of Fuel to materialize a stream, keeping a reference to this object graph, and then loading the new version of Fuel and serializing the object graph back to a file. We include below an example of such a format migration. Let's say we have some files serialized with Fuel 1.7 in a Pharo 1.4 image and we want to migrate them to Fuel 1.9. @@ -591,6 +593,7 @@ fileNames := #('a.fuel' 'b.fuel' 'c.fuel' 'd.fuel' 'e.fuel'). objectsByFileName := Dictionary new. (ConfigurationOfFuel project version: oldVersion) load. +"Need to do it like this otherwise the class is decided at compile time." materializerClass := Smalltalk at: #FLMaterializer. fileNames do: [ :fileName | @@ -599,6 +602,7 @@ fileNames do: [ :fileName | put: (materializerClass materializeFromFileNamed: fileName) ]. (ConfigurationOfFuel project version: newVersion) load. +"Need to do it like this otherwise the class is decided at compile time." serializerClass := Smalltalk at: #FLSerializer. objectsByFileName keysAndValuesDo: [ :fileName :objects | @@ -607,38 +611,42 @@ objectsByFileName keysAndValuesDo: [ :fileName :objects | toFileNamed: 'migrated-', fileName ]. ]]] -@@note Note 1: We assume in this example that the number of objects to migrate can be materialized all together at the same time. This assumption may be wrong. In such case, you could adapt the script to split the list of files and do the migration in parts. - -@@note Note 2: It is necessary to fetch the classes in the System Dictionary after the desired Fuel version has been loaded. +We assume in this example that the number of objects to migrate can be materialized all together at the same time. This assumption may be wrong. In such case, you could adapt the script to split the list of files and do the migration in parts. -@@note Note 3: This script should be evaluated in the original image. For example, we don't guarantee that Fuel 1.7 loads in Pharo 2.0, but we know that Fuel 1.9 loads in Pharo 1.4. +@@note This script should be evaluated in the original image. We don't guarantee that Fuel 1.7 loads in Pharo 2.0, but we do know that Fuel 1.9 loads in Pharo 1.4. %=========================================================================% -!! Debugging +!! Fuel Development: Debugging Tools + +There are a couple of packages that help us to debug Fuel, and they are loaded as follows: + +[[[language=Smalltalk +Gofer it + url: 'http://smalltalkhub.com/mc/Pharo/Fuel/main'; + package: 'ConfigurationOfFuel'; + load. + +(ConfigurationOfFuel project version: #stable) + load: #(FuelDebug FuelPreview). +]]] -There are a couple of packages that help us debugging Fuel. To understand the output of the tools, you should know some basics of how Fuel works internally. +We now first talk about the internals of Fuel before we present the different tools: a graph viewer, a serialization logger and a materialization logger. -!!!Serialization -The most important thing to know is that serialization is split in two main steps: analysis and encoding. +!!!Internal Workings -!!!!Analysis -The analysis phase consists of walking the graph from the specified root object and mapping each traversed object to its corresponding groupi, called a -""cluster"". +To understand the output of these tools, you should first know some basics of how Fuel works internally, both for serialization and materialization. -!!!!Encoding -After analysis, we write the graph to the stream linarly, in these steps: +;Serialization +:The most important thing to know is that serialization is split in two main steps: analysis and encoding. The analysis phase consists of walking the graph from the specified root object and mapping each traversed object to its corresponding group, called a ""cluster"". The encoding phase runs after analysis, and it writes the graph to the stream linarly, in these steps: #header #for each cluster, instances part #for each cluster, references part #trailer -!!!Materialization -Because of the extra effort put into serialization, we can materialize the graph in a single phase. - -!!!!Decoding -We decode the graph by reading the input stream linearly, in the same order it was written. The materialization steps are obviously analogous to the ones above: +;Materialization +: Because of the extra effort put into serialization, we can materialize the graph in a single phase. We decode the graph by reading the input stream linearly, in the same order it was written. The materialization steps are obviously analogous to the ones above: #header #for each cluster, instances part @@ -649,24 +657,8 @@ It is important to understand that references are ""not"" stored together with t stored together, after the references. We use this to materialize all the references in a single step, when we know that all the objects have already been materialized. -!!!Debug Tools -Ensure you have them with: - -[[[language=Smalltalk - Gofer it - url: 'http://smalltalkhub.com/mc/Pharo/Fuel/main'; - package: 'ConfigurationOfFuel'; - load. - -(ConfigurationOfFuel project version: #stable) - load: #(FuelDebug FuelPreview). -]]] - -Here is a list of some useful class comments. - - -!!!!FLGraphViewBuilder -I add draw capabilities to analysis in ==FuelDebug== package. +!!!FLGraphViewBuilder +This class adds drawing capabilities to the analysis in the ==FuelDebug== package. Right-click a node to inspect it. Some examples: @@ -695,7 +687,7 @@ Figure *@figFuelPreview* shows how they look like. +Visual preview of graph to be serialized>file://figures/FuelPreview.png|width=60|label=figFuelPreview+ _ -!!!!FLDebugSerialization +!!!FLDebugSerialization I am a serialization which facilitates debugging, by logging the stream position before and after main steps of ==FLSerialization==, including cluster information. Obviously, you should be familiar with the algorithm to understand the output log. @@ -714,7 +706,7 @@ Then, inspect the output log: FLDebugSerialization last log. ]]] -!!!!FLDebugMaterialization +!!!FLDebugMaterialization I am a materialization which facilitates debugging, by logging the stream position before and after main steps of ==FLMaterialization==, including cluster information. Obviously, you should be familiar with the algorithm to understand the output log.