-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Making rgl widget #141
Comments
@dmurdoch Great to know that you want to make an
You can use |
The one challenge I see here is that <!-- ****** spheres object 6 ****** -->
<script id="vshader6" type="x-shader/x-vertex">
attribute vec3 aPos;
attribute vec4 aCol;
uniform mat4 mvMatrix;
uniform mat4 prMatrix;
varying vec4 vCol;
varying vec4 vPosition;
attribute vec3 aNorm;
uniform mat4 normMatrix;
varying vec3 vNormal;
void main(void) {
vPosition = mvMatrix * vec4(aPos, 1.);
gl_Position = prMatrix * vPosition;
vCol = aCol;
vNormal = normalize((normMatrix * vec4(aNorm, 1.)).xyz);
}
</script> Injecting such a |
That's not really necessary, it's just the way the model code did it that I based writeWebGL on. I was planning to make it pure Javascript anyway: I just need to put that shader code in a string. |
Good to know @dmurdoch. If you hit any roadblocks along the way of implementing this widget, do let us know. |
I have some questions about details and philosophy of the implementation of htmlwidgets. Some background first: Currently, writeWebGL is monolithic, not distinguishing between initialization and rendering. So I need to decide where the dividing line goes. My plan for the first pass is to mimic writeWebGL from the user point of view. The user will call a series of rgl functions creating a scene within R, then make a single function call to translate everything into WebGL for display in the widget. Next I'll add functions to allow modifications to an existing scene, but not adding or deleting elements. These will be like the existing propertySetter() and related functions. Finally I'll make it general, so the htmlwidget display in RStudio could be used as the main display while developing the scene. So, what's initialization, and what's rendering? One idea is a very minimal initialization: just create an empty rglClass object. When renderValue is called, argument "x" will be the whole scene to be rendered. The other extreme is that the initialization does most of what writeWebGL does now, and the renderValue function hardly does anything. Which makes sense? And one other question: in RMarkdown documents, it makes sense that each new display can start with the previous one, and just make some modifications. How do I do this in htmlwidgets? |
(Sorry for brevity, typing this from my phone while on vacation) Initialize is called once, renderValue can be called many times (not in an RStudio or R Markdown context, but when used with Shiny). If new data should always cause your widget to be reset then essentially everything in your monolithic approach can go into renderValue and you can leave initialize empty. However if you want to, say, preserve the camera angle (or whatever it's called in WebGL) or you want to somehow diff the old and new data and only change the parts of the scene that are necessary, then you might need to do a different initialize/renderValue split. For the second question, we do this by having the R representation of the widget be a list that you modify using accessor functions, with pass by value semantics. The dygraph and leaflet packages both use this approach. |
Examples of how progressively building up a leaflet visualization are here: http://rstudio.github.io/leaflet/. You'll notice two functionally equivalent syntaxes. Here's the traditional one passing the map as the first argument to various mutator functions: m <- leaflet()
m #print empty map
m <- addTiles(m)
m # print map with tiles
m <- addMarkers(m, lng=174.768, lat=-36.852, popup="The birthplace of R")
m # print map with markers The same transformations can also be done via the leaflet() %>%
addTiles() %>%
addMarkers(lng=174.768, lat=-36.852, popup="The birthplace of R") As Joe described there are different philosophies on whether to rebuild the world in |
Thanks Joe and JJ. Since rgl scenes are so big, I think it makes sense to make the initialize method fairly heavy, and then each of the renderValue calls won't need to send much. Now I have a more detailed question. (Remember that I don't really know what I'm doing in Javascript.) I need to transfer 4 by 4 matrices from R into Javascript "CanvasMatrix4" objects. I have been doing it by generating code like m = new CanvasMatrix4(); Can this happen automatically in JSON (e.g. with a custom serializer), or should I just pass an array, and convert it to a CanvasMatrix4 object with Javascript code like the above? |
The latter. htmlwidgets will only give you generic JS scalars, arrays, and objects--it's entirely on the author to turn those into domain specific objects (if necessary). |
One more thing to keep in mind: the distinction between In the Shiny case however the widget is updated in place via the call to |
I've looked at it in more detail, and now I have an approach in mind. The initialize call will just set up an empty scene. When I call renderValue, it will either send the whole scene, or just some updates, according to a flag in x. (Essentially the flag will say whether to clear the scene or not.) I don't want to send the whole scene every time because that can be a lot of data, and I'm worried it will be too slow. My first attempt at a Shiny implementation (back in December, long before I'd heard of htmlwidgets) resent the whole scene, and it took a couple of seconds to make a change. |
Not to complicate things too much further, but I think the direction you are going is very similar to what we ended up with for leaflet. In that case for the Shiny scenario we created a If the idea is to avoid the transfer of data from R to JavaScript then I think you'll need to do something like leafletProxy to avoid re-sending all of @jcheng5 and @yihui Know more about the particulars of the proxy pattern so may want to add something here. |
Thanks for the pointer. It's the transfer of data that's the issue; WebGL is quite fast at rendering once things are in the browser. I'll follow the leafletProxy model when I get to that part of the implementation. |
I'm making some progress on this, but now my lack of Javascript knowledge is slowing me down. These are probably simple questions:
|
There are functions that allow you to do these operations. For (2), you can use the code used in d3.merge = function(arrays) {
var n = arrays.length,
m,
i = -1,
j = 0,
merged,
array;
while (++i < n) j += arrays[i].length;
merged = new Array(j);
while (--n >= 0) {
array = arrays[n];
m = array.length;
while (--m >= 0) {
merged[--j] = array[m];
}
}
return merged;
}; So As for (3), the approach you are using is correct. You could wrap them up in to functions I will try to add a solution for (1) later today. |
For (1), I think you can use the x = [[1, 2], [3, 4]]
y = [[5, 6]]
x.concat(y)
console.log(x) This will give x = [[1, 2], [3, 4], [5, 6]] |
Ok, deleted comment also |
I'm making good progress on this: most types of object now display. I'm currently working on textures. I think I could get them to work similarly to the way rgl handles them (i.e. copy the image file into the same dir as the web page, load from there), but a better way would be to put the image directly into the .html source. Is there support for that in htmlwidgets? Specifically I need to end up with a Javascript Image() object that contains an image from a file, but I don't want the Javascript to read the file, I want R to read the file. |
There is also a pending PR #83 that attempts to make this process a whole lot easier. I will review it this weekend to see if we can merge this soon, as it would make things much easier. |
I think my question was more naive than you thought. I've found the knitr::image_uri function, and it does what I wanted. For anyone coming along later: In R, do
which puts a long string representation of the image into a field of the object to display. In Javascript, turn it into an image by
That's all that's needed. |
That's great, glad it was that simple! On Sat, Aug 22, 2015 at 8:57 AM, dmurdoch [email protected] wrote:
|
I've now got a mostly working version of the rglwidget, in a package of that name on R-forge. It requires an unreleased version of rgl, also on R-forge. Long range plans are to backport many of the rglwidget changes to rgl, but that won't happen for a while. In the meantime, I've got one problem I'd like to work on. I think if I just display my rglwidget in the RStudio console, the viewer should show the display, but it doesn't. Is there anything like the Firefox web console or debugger that I can use to see what's going wrong? |
On Linux and WIndows, you should be able to right click on the widget page, and use |
Thanks. I'm mainly using 0.99.467 on Mac OSX, so no Inspect there. I can run Windows XP in a VM, but current RStudio doesn't work well in XP. The Inspect page did work, but gave some spurious syntax errors. I can also run Ubuntu in a VM. I'm just installing 0.99.467 there. |
I've basically got the rglwidget working in an Rmarkdown document now, so I wanted to get it going in Shiny. I've got two questions. The last time I tried to do this (about 10 months ago, with rgl::writeWebGL writing the Javascript), I found the response time was too slow. I think this was because rgl scenes can really have a huge amount of data in them (up to megabytes), and the way I wrote it, this would all have to be downloaded again if the user made a change via Shiny. I want sliders that give near-instant responses. Earlier this year I wrote various functions that took an existing scene and just modified a few The time lag may be less of a problem now with the rglwidget since the Javascript doesn't get re-sent with each change, but there's still a lot of data. I decided on the following approach to deal with it. If I want Shiny to control a scene without redrawing the whole thing, I would insert two output objects: one that draws the scene, and the other that receives commands from Shiny and modifies the first one. First question: is this design a bad idea? Assuming I go with the above design, the rglcontroller needs a way to refer to the rglwidget it is controlling. I had assumed I could use elementId for this, but I'm getting warnings:
Second question: how should one Shiny output object refer to another one in Javascript? |
Great! A couple of things that are relevant here:
|
I think this may be where you need the "proxy" pattern implemented by On Sun, Aug 30, 2015 at 1:26 PM, Ramnath Vaidyanathan <
|
Thanks, I've got it working now with the rglcontroller. Now I need to write some controllers so I can try it out. I'll also take a look at the leaflet source to see if that way makes more sense. |
In a knitr document, I may have several variations on the same rgl scene. Using the old writeWebGL scheme I could figure out the name of the early one and copy some data from it in the later ones. I'd like to do the same with the rglwidget implementation. I can do it by explicitly specifying the elementId of each widget, but it would be nicer if the code could work out the auto-generated names that knitr will use. Is there a way to figure out either the next auto-generated name, or the last one that was used? |
I think that the explicit elementId is currently your only recourse (here's It wouldn't be too difficult to provide a bit more transparency here by On Sat, Oct 17, 2015 at 12:32 PM, dmurdoch [email protected] wrote:
|
I tried setting the elementId automatically, but then Shiny gives a warning. Is there a way to detect that my rglwidget() is being called from a Shiny server? |
Are you generating elementId automatically on the client side (in javascript?). If so, you can check for the presence of |
FYI, |
Thanks @yihui |
Thanks, but I think I need this on the R side. I'm keeping information about previous instances of the widget so that I don't have to send repeated data in later instances. I need to consult that list before constructing x in the createWidget() call. |
You can try |
@jcheng5, thanks, that works for me. |
If I have an rgl scene in a widget, I might want to add some controls for playing an animation, etc. I have two ways to do that now:
I don't have a way to do this in the RStudio viewer, since (as far as I know) I can only display one widget at a time there. So, my questions:
|
One approach to this would be to create an R function that returns a "Shiny http://rmarkdown.rstudio.com/authoring_shiny_widgets.html Basically you create an R function that returns a Shiny application which The downside of this is that the R console is blocked while your "gadget" On Fri, Oct 30, 2015 at 1:46 PM, dmurdoch [email protected] wrote:
|
A few months ago (in pre-widget days) Henrik Bengtsson suggested I should allow users to add classes to the WebGL that writeWebGL inserted, to make it easier to do custom CSS. Now with the widget, I see the class is automatically set to my widget name plus "html-widget-static-bound". Is there a way for users to add other class names? If not, this might be a useful addition to createWidget. |
I think the best way to do this is allow specification of the extra class On Sat, Nov 28, 2015 at 9:12 AM, dmurdoch [email protected] wrote:
|
Yes, I could do that pretty easily. The reason I thought it might be more appropriate in createWidget was that it's not really an rgl-specific request -- people might want to customize other widgets too. I already pass ... on to createWidget, so I'd automatically inherit this functionality if it was there. But it's certainly not a big deal. All widget instances have ids, so their display could be customized via the id. |
As @jjallaire said, this would have to be done from the javascript side, since the widget html does not receive the data passed on. We could add this behavior to |
I think it's actually in R: my widget_html function, called from htmlwidgets:::widget_html, receives a "class" argument. I think it contains just the widget name, the other name is added later. The suggestion was to allow users to add more names to the class via an argument to createWidget. But again, it's not important. |
@dmurdoch Regarding the question you brought up on Oct 30 (about showing multiple widgets in the viewer and having them interact, without Shiny) you'll be able to do this soon. You can already show multiple widgets in the viewer but it's currently difficult to get them to interact: htmltools::browsable(tagList(
widget1,
widget2
)) You can put arbitrary htmltools tag objects in tagList, so you can do layouts with divs, tables, or whatever. The trouble in getting them to interact is that there's not a natural notion of "JavaScript object that represents widget x", and even if there were, there isn't a way to say "give me the JS object that represents widget x" from any old snippet of JS. These should both be coming next week; keep an eye on #171. |
@dmurdoch With regards to the custom CSS classes, I envisioned that custom CSS classes would be added to your own (nested) elements in JavaScript. I wouldn't generally recommend you adding them to the outermost |
@jcheng5, thanks for the multiple widget pointer. In fact, that works for me right now: the normal way my rglwidget interacts with the controller is by putting an explicit elementId on the rglwidget. I can do that with your code. For example,
This probably needs the R-forge versions of rglwidget and rgl; there have been a lot of changes lately. |
@jcheng5, I've noticed one strange thing: if I display an rglwidget in a browsable tagList as above, and the rglwidget has some semi-transparent components (e.g. by adding "alpha = 0.5" to the spheres3d() call in that example), then the rglwidget doesn't display properly at first. Once I move it, things are fine. Displaying the same rglwidget on its own is fine. Is there some difference between the order of initialization for a widget being displayed on its own and one being displayed as part of a browsable tagList? |
@dmurdoch: Hmmm, that's quite surprising. If you View Source in a browser, does the generated HTML look different? Especially the |
|
On 02/12/2015 12:39 PM, timelyportfolio wrote:
Thanks for the suggestions. I'm not going to have a lot more time today Duncan |
I'm seeing the same error elsewhere, it has nothing to do with the tagList(). My previous test was flawed. So this is purely my error. Sorry for the noise. |
I think I understand the problem now, and have mostly fixed it. A different question: I'm trying out magrittr pipes as a way to construct more complicated displays, e.g.
I'll want these to handle objects produced by
Are there any others? Is there an existing convention in other widgets for doing this (i.e. an arg name for the receiver?) I'd like to be able to do something like
|
There isn't really a convention for this yet (that I know of) as we're just I was imagining not combining different widgets with %>% but rather with htmlPage( I don't imagine that mere aggregation of these widgets is going to On Mon, Dec 7, 2015 at 6:02 AM dmurdoch [email protected] wrote:
|
On 07/12/2015 12:59 PM, Joe Cheng wrote:
Thanks. The main advantage I see to using the %>% notation is that my Duncan |
OK, sure, that makes sense. If you want you can now retrieve widget instance objects by class ( |
I see the advantage of using |
I'd like to make an rgl widget, but I'm having trouble getting started, because the examples I've seen start with a Javascript library, rather than starting with R code. Here's what I'm thinking of doing:
Does this sound like the right approach? Are there any examples to work from that are similar?
Duncan Murdoch
The text was updated successfully, but these errors were encountered: