CCfits (and) starts and GDAL isolation

Rickard & Michael S.: Last week we mainly focused on the rendering of rover-derived models. As of now we can render them, but without a correct placement. We’ve only started looking into the placement of the models. We’ve also received new, more accurate data for rendering the rover path on top of the reference ellipsoid, and implemented it in OpenSpace.

Michael N. & Oskar: Last week we integrated the CCfits library into OpenSpace as a separate module. The module will be used to read and process the high resolution imagery in the FITS format that the spacecraft provide. We’ve also started adding kameleon’s field line tracer to OpenSpace in order to support more of CCMC’s data models and managed to render some of the data from ENLIL. CCMC also provided us with the SPICE kernel for SDO and we can now render its trail/orbit in the correct location.

Jonathan: Since last week I’ve implemented a TouchInteraction class that handles and interprets the input. Zooming, rotation and panning now work by having the touch event add a velocity, and the interaction class calculates the new camera position orientation by incrementing with velocity times dt. The velocities all have some friction/intertia to eventually stop the interaction and make the movement smooth. In the coming week I will continue to work on how the touch input is best interpreted, as well as adding rolling to the multi-touch gesture list.

Kalle: I have worked on a memory aware tile cache that can be dynamically resized, to limit the amount of memory used for caching tiles. I have also isolated GDAL in the globe browsing module so that GDAL is no longer a necessary dependency. If OpenSpace is not compiled with GDAL, a simple tile reader will be used instead. The simple tile reader currently only supports the common image formats used by the texture loader, and the textures currently need to have width and height being powers of two.

Gene: The frequent crashing of the Jenkins Mac node seems to be fixed now, but I’m watching it long-term to make sure it stays that way. I fixed a build problem with satellite TLE downloading with luasocket. I can now get the luasocket library to build as a part of building OpenSpace, rather than having to build it separately. I’m working on a feature that allows scenegraph nodes to be tagged and then operated on (e.g. renderable commands) as a group.

Field Lines and Rover Views

Gene: I got the ‘luasocket’ library working for automatically downloading latest TLE files for satellite telemetry. Testing of this feature on all 3 platforms is needed. I created a short demo video of the satellites scene that I’ll share in order to solicit feedback.

Oskar & Michael N.: We’ve started implementing time varying field lines which include a visualization of the direction of each field line. Currently it’s all being pre-processed and we are waiting for a larger dataset to be able to improve it. We’ve also started implementing a class for projecting photographs taken from spacecraft onto an image plane within their field of view. The image plane is movable within the frustum of the camera that took the image.

Matt: I’m hunting down some build bugs that are preventing me from getting the latest develop branch updates. I’m concerned they might be a larger Heisenbug issue than just my machine configurations.

Rickard & Michael S.: We’ve mainly worked on two different functionalities regarding the rover terrain. One is being able to view rover derived panoramas (much like google street view) by projecting the image on top of a sphere and placing the camera in the center of the sphere. The other part is visualizing the path of the rover on top of HiRISE height maps. There are still some work on both parts to make it look super good but the base functionality is there.

Jonathan: Since last week I’ve looked through and tested different ideas on how to store the input more efficiently. The pinch-to-zoom feature now works pretty well after resolving a bug that was introduced by not checking if we got any new input from the last frame. Coming week I will focus on an interaction class for the touch input and to do rotations with spherical coordinates rather than angles. There’s also an error in the module’s CMake files that I will look at to resolve failed builds on Linux and macOS.

Eric: I started a script to be run on the Jenkins server after a build to pack up binaries and some data (either a minimal subset or a larger collection) for distribution to users. I just used a shell script to create a tarball and/or zip file, but Alex has pointed me to CPack, a cousin of CMake, which could make it much easier to do this on all platforms, so I’ll be investigating that next.

Kalle: I solved a couple of issues related to tile rendering for globes. A bug previously caused some textures which did not have the number of bytes per row divisible by four to get corrupted once uploaded to the GPU. I also started working on a memory aware tile cache to prevent memory overflow when allocating too many textures for tiles.

Alex visits North Carolina

Alex: This week, I was invited to the North Carolina Museum of Natural Sciences, one of our Informal Science Institution partners, to install OpenSpace on their systems. It was a full success and is now installed in the science panorama and visible to the general public in the Astrolab. On the code side of things, I made the specification of SGCT configuration files a bit simpler by enabling the use of Lua scripts to generate SGCT configuration files on the fly, thus making a resolution change a matter of changing arguments.

Kalle: I just got back in OpenSpace development. I have looked at the current issues and started planning and figuring out where I will be needed. I will probably continue doing some work on the globe browsing feature that me and Erik did not have time to do during our thesis work.

Gene: This week I have been working on debugging problems with FreeImage library on linux, and worked on building the Lua socket library into OpenSpace’s Lua interpreter.

Michael N. & Oskar: We’ve been discussing the details and priorities of the Solstice event in June with the people at CCMC. We managed to render the trails of the SDO and STEREO satellites and been digging into some of the already existing work related to field line tracing. We will start to look at the already supported BATS-R-US model and get that to work with time-varying data before examining other models related to coronal magnetic fields such as the PFSS.

Jonathan: I’ve managed to implement some crude multi-touch interaction with zooming and changing the orientation of the camera. The methods use will need to be reiterated to feel more intuitive, my next step is to have the touch callbacks manipulate the view with velocities and inertia rather than absolute positions.

The New Crew Gets to Work

Alex: This week has been mostly focussed on the supporting backend. First and foremost, we now have a Python script that will check the files in the repository for consistency (kind of like baby’s first linter), which will hopefully aid in detecting some bugs in the future. The second big part is the addition of callback functions in the main loops to enable a greater degree of customization from modules. This will help in removing the module dependency inside libOpenspace, an important step towards transitioning to dynamic libraries. Finally, I added code to make the capabilities of the system queryable from Lua scripts. This will be very useful to customize scene files, for example enabling the loading of different texture resolutions depending on the capabilities of the hardware.

Gene: I got the satellite branch working with TLE “batch” files downloaded from celestrak.com. After a lot of testing, I found & fixed a bug in the satellite telemetry code that affected position accuracy. I have continued to work on trying to get a working GL texture library in linux. deVIL doesn’t work, FreeImage doesn’t fully work, and SOIL loads all textures but needs image correction. I’ve also worked on getting the cURL module incorporated into OpenSpace’s built-in Lua interpreter.

Matt: I started implementing our configuration system for Launcher specific properties this week.

Eric: This week I made some small changes to the Launcher synchronization controls. Only the ‘default’ scene is selected by default, so the user must check the boxes to download more than the minimum. The checked boxes are green, while unchecked boxes are grey (not red). In the future we might include some status information in this window, but for now it’s just to select what gets downloaded. Matt and I are going to tackle another problem with the sync function: not downloading files that have already been downloaded. I’ve also started thinking about how to pack up binaries for distribution, and of course this will be an automated process.

The next group of Masters students are now getting to work on their projects, and we’ll be hearing much from them in the future about their plans and progress. Here is the first round:

Michael N. & Oskar: Last week we arrived at CCMC where we will work on visualisations associated with Space Weather. This will amongst other things involve expanding the visualisations for Earth’s magnetic field. During our first week we got introduced to the team at CCMC and sat in on several meetings and seminars. We also got everything up and running and continued looking at the code base and some example data provided by the CCMC. Michael also debugged the image loading issues on Linux. It seems that SOIL cannot load progressive JPEGS (baseline is preferred). Latest FreeImage requires JPEG-library version 80, whereas libjpeg-turbo (dependency in SGCT) seems to use version 62.

Jonathan: I arrived to Salt Lake and the SCI Institute three weeks ago and have since then been working on developing a multi-touch interface. I’ve implemented a new touch module that makes OpenSpace TUIO-aware, a cross-platform touch protocol for tangible multi-touch surfaces. Now my main focus is to continue with developing touch gestures to interact with the application through zooming and rotating.

Michael S. & Rickard: We arrived at AMNH, New York about three weeks ago. We will be working with extending the current globebrowsing module with a close up interaction mode of models created from the images from the Mars Rovers. During our first weeks here we have familiarized ourselves with the codebase of OpenSpace and the globebrowsing module. As for now we are doing pre-studies regarding model rendering and looking into alignment issues regarding the models and HiRISE heightmaps.

Getting Down to Work

Emil: Lately, I have been focusing on developing the open source app C-Troll (https://github.com/c-toolbox/C-Troll) that will make it easier to launch and control applications such as OpenSpace on clustered environments. The application is split up into three tiers: A “core” application to run on a master node, which sends out commands to the slave nodes. Slaves run a “tray” application which receives commands, launch and quit applications as subprocesses, and collect log messages and send them back to the core. The third tier is a web application (which for example can run from a tablet) connected to the core application, which can be used to view log messages.

Alex: After a one-month hiatus of coding due to moving to a new city, I could finally return to it and spend my first week on fixing various issues. I started cleanup of the globebrowsing branch to make it look and feel more like the rest of the code base. In addition, we now have a script that will check some of the style guides for all files to maintain a consistent look to the include guards. Regarding non-coding events, I presented OpenSpace to an interested group at the MIT Media Lab, as well as my new home, the Center for Data Science at NYU.

Matt: After a first semester focused on coursework, I’m excited to be back on the project! I’m starting to dig into the multi-resolution volume rendering work done by Emil and Tomas so we can expand it, and I’m trying to improve the Launcher’s syncing process with more controls and validation.

Gene: I’m currently working on automatic download of satellite telemetry (TLE) files from celestrak.com. This will allow users to run with the latest satellite data. I also spent a little time troubleshooting a problem with the FreeImage library on linux. This is what reads textures for planet surfaces and other things. The SOIL library works on linux, but has an image-flipped problem that was corrected with FreeImage. I worked with the VR port of OpenSpace a bit. I created separate SGCT cfg files for the HTC Vive and Occulus Rift, since they appear to use slightly different render target resolutions. The OpenVR wiki page was updated with this info.

SCI Utah!

On February 2nd and 3rd a group of OpenSpace developers and project leaders met at the Scientific Computing and Imaging Institute (SCI) at the University of Utah to plan out our activities for the coming year.

Over those two days there was a lot to discuss. After hearing reports on current status of functionality, software, and site reports, we had progress reports on specific developments for the past year, specifically dynamic scenegraph rendering (Emil), globe browsing (Alex), and atmospheric effects and integration of the Digital Universe catalog (Jonothas). Alex and Eric reviewed our development process and branching practices. We then had a productive discussion of GUI design, along with demonstrations of the GUI in two related products, DigiStar and UniView.

On the second day we had a demonstration of OpenSpace running on the PowerWall at SCI (and celebrated the fact that we’d found and fixed the cause of recent performace issues). In the afternoon we all got together to map out the work for the year, broken down into “Milestones” (often associated with future events) and the “issues” needed to achieve these milestones. These are now all visible on our GitHub site. There’s now lots to do, and we’ve all started doing it.

Getting started in the New Year

It’s been a bit quiet in OpenSpace development with the holidays and people traveling and such, but some progress has been made.

Emil completed the main part of his work on refactoring the SceneGraph API and pushed the branch to GitHub for testing. Unfortunately it doesn’t build on Linux and Mac yet, so some small tweaks are probably needed to get it ready to pull into the develop branch for everyone to enjoy. That should follow soon.

Gene has continued to investigate some performance issues on the multi-display wall (the “Power Wall”) at SCI. There are time synchronization issues and low framerates with the OSIRIS-REx scene (all other scenes are running well, however). He also updated the wiki page that discusses scene graph nodes. His intent here was to add more information to help those getting started with a new scene. At the same time, he put in a lot of references to the auto-generated documentation, so that the information does not become out-of-date as the code changes.

Alex has moved to New York City to begin his new position as a post-doc at NYU.

Eric has been busy with end of semester grading, followed by preparations for the next semester, which has already begun.

All of us have been preparing for the yearly collaboration meeting in February, this year at the Scientific Computing and Imaging (SCI) Institute at the University of Utah, even if not everyone can attend in person.

We are also all looking forward to the next batch of students who will be joining the team soon. More information about them and the projects they will be working on will be posted here soon as they get started and get up to speed.

Ending the old year, getting ready for the new

Alex: The “develop” branch on github has been pulled into the “master” branch, creating a new stable beta release, with version number 0.6.0 and labeled “prerelease-11 (AGU)”. Carter used this code yesterday at the meeting of the American Geophysical Union this week in San Francisco. So far this is just a change to the branch, but at the end of the month, or sometime next month, there will be executable packages created for distribution.

Gene: most recently I’ve been running globebrowsing with the Mars CTX data served up from here at SCI. I experimented with different configurations and on different platforms, trying to figure out why the performance was slow running on Windows here. After getting this resolved, I updated the wiki pages for the different platforms with this information.

Emil: I’ve been working on refining the SceneGraph API to protect against mistakes in distance and scale. This looks like it will be an overall refactoring of the SceneGraph code. Right now I’m looking at how Scenes are loaded, which is done in the SceneGraph class. It’s fairly monolithic, and it looks like it will be tricky to reload a scene, or to do multithreading, but I’d like to make it more flexible.

Jonathas has continued working on the new atmospheric modeling code. Although nothing has been checked in to github in a long while, he’s been keeping his own backup of his progress on DropBox.

Erik and Kalle defended their thesis and gave a public presentation of their work. They are done! Congratulations!

After the Buildathon

Alex: So many things have happened since the last blog post! Apart from restructuring the way scene graph transformations are reported in the GUI and the performance measurements, there has been some more work on providing documentation. Every key binding now has an optional user-facing documentation describing what the key does. Combining this with the automatic documentation generation will make it much easier for end users to use the software.

OpenSpace Builders Community

Eric: After the Buildathon I collected the results from both teams and put them in new repositories on github (OpenSpace/team-cassini and OpenSpace/team-messenger), and I added interested Buildathon participants to our github community so that they can continue working on their projects, if they wish to do so. The Slack that we set up for the Buildathon has been renamed to “OpenSpace Builder Community” to serve as a communications hub for what we hope will be a growing builder community.

Gene: This month I spent most of one week preparing for SCIx, which was an event we held here at the University of Utah. This was timed to coincide with the Supercomputing conference in downtown Salt Lake City. We had visitors from that conference, from industry, and other departments at the University. We showed demos from the OSIRIS-REx mission and talked to visitors about OpenSpace. Since then, I’ve gotten the Jenkins node for the Mac up and running. There was a problem with its CMake build plugin, so I reproduced the build steps manually as a workaround.

Erik & Kalle are still working hard on writing their thesis.

AMNH Buildathon ’16

On Saturday, October 29, 2016 a group of 21 “builders” met all day in a classroom at the American Museum of Natural History (AMNH) in New York City for the 2016 OpenSpace Buildathon. Participants divided into two teams to put together OpenSpace scenes for the Cassini mission to Saturn and the Messenger mission to Mercury. The OpenSpace development team actively supported the Buildathon, and the developers are very pleased with the results.

Buildathon participants started learning about OpenSpace in some detail the week before the event, as many of them attempted to compile the software on their own computers, on either Windows, Mac, or Ubuntu Linux. Members of the developer team helped guide the new builders through the necessary steps via a Slack site that had been set up for the event. One result of this was that the documentation for compiling OpenSpace was improved significantly, especially the page specific to Windows. Having this kind of feedback from outside users of the code was very helpful.

The developers also actively supported the builders on the day of the event, primarily via Slack. Alex, Emil, Gene, and Jonathas were all engaged on Slack with builders learning about the inner workings of OpenSpace configuration files, SPICE kernels, and orbital mechanics. Eric was there in person and helped pivot the builders from “building” as “compiling” to “building” as creating content for their team’s mission, and at one point he also distributed some necessary code via a USB stick when the server it was hosted on bogged down. Direct assistance from the development team was a key reason for the overall success of the event. Alex is especially to be commended: at one point he was carrying on multiple conversations with different builders in different Slack channels, and he also stayed up past 1:00 AM in his local timezone to make sure the builders were on the right track.

Overall, the Buildathon was a success in several ways. It helped the developers improve the documentation and to see the software from a beginner’s eyes, and it tested the software in real use by non-developers. It also produced two new missions, which will be added to OpenSpace. The content created by both teams has been put in two new repositories on github (team-cassini and team-messenger), and several of the participants have been added to those repos so that they can continue to work on the missions if they wish. There is also now a video of the final presentation of the Messenger mission on Youtube: AMNH Buildathon ’16 – Team Messenger.

1 2 3 4 5 6 7