Beginning Development Once more

It has been almost a year since we started this project and for the last 9 months I began work on another project called Briefcase. But now that the summer is almost here again development will begin once more. Many of the bugs that existed in the last version will be fixed and we will start writing more default plugins for you to use. Possibly also create a create a compiled version of the python plugins so that they are less noticed in the overall system. I have learned a lot since I last worked on this project and there are many things to fix and make better. Gabe will be going off to work for Hasbro soon so a new person, James Kalfas, will be joining me on the fronted development. This will be fun, I am looking forward to developing this application once more.

– Asher

Creating Secure Connections

One of the main focuses of Olympus is to be able to access any number of computers securely. To do this we will be using a RSA style key-pair to securely send and receive data between client and server. The source server has a list of all of the remote server’s public keys (the pubkeys are also used a unique identifiers to make each computer unique, independent of its IP address). When a new computer is added to the list the new remote computer will send its public key over to the source server. The sys-admin can then use a fingerprint to make sure that the key is correct, thus creating another secure connection. The communication between the web client and the source server will eventually be an ssl connection however for now we are still working out a few bugs with the encryption method and the certificates.

Saving configurations using the filesystem

Olympus is meant to be loaded in parts, allowing for many queries to be made to it at once without noticeable delay. This is achieved by having each element of a the configuration, for example a list of servers, stored using the file system instead of a configuration file. This allows us to load each individual file about the server without having to load the entire set of information about every server. As a result this will allow for the program to be completely scalable to hundreds or thousands of servers without slowing down the program much on runtime, though we are not able to test more then ten at this point in time. Another benefit of this system is that if a system admin wanted to copy a server configuration from one Olympus program to another it would be a simple matter of copying a file or folder over, and would not require any kind of configuration change within Olympus. If you wanted to write a program that messes with the configurations of servers or plug-ins it would be a simple manner of changing the contents of the files, though that it not thread safe yet.

To maintain file integrity and avoid serious race conditions the process of writing to these files is contained in a single core while the process of reading from these files is un-restricted. In the future this method will change to allow for multiple files to be written to at once without fear of corruption. This will be done using file write locks to prevent a file from being opened twice.

Thats all for now

Promoting an Open Source Project

As completion of Olympus draws near its time to think about how to promote the project.  As a developer, one may be thinking that at the end of the project their work is finished.  This couldn’t be further from the truth.  To be a responsible citizen in the open source community one must not only create a project but then share it with the rest of the world.  Now how to do this exactly can seem like a project in and of itself, but after some quick research one finds that this is a common question among developers in the community.

After looking into it ourselves and getting tips from fellow open source members, we came up with the following Open Source Project Promotion Plan:

Step 1: Add project to a code repository –  We use github.  There’s also SourceForge and Google Code.  Take your pick.

Step 2: Add project to freshmeat – Another great software hub

Step 3: Maintain a blog and wiki for the project – its good to have a blog (like this one) and at the very least you can use github’s wonderful wiki feature

Step 4: Add Screenshots of your project – its true that pictures say a thousand words, screenshots of your project are a necessity for success

Step 5: Make it easy to install – if you want to make it popular make sure its easy to get running.  People want software that helps, not a battle.  For this we are looking into packaging for some popular Linux distributions.  For Windows you might look into NSIS

Step 6: Spread the word – Let your friends know about it. Use social media.  If you can, make a video of your project in action and put it on Youtube.  Just remember that the completion of your code is not the end of the project, its the beginning.

Writing Our Own Webserver Cont.

After much effort and debugging the UNIX (POSIX) version of the webserver is finally bug free enough to consistently send data across. As it is configured now you can only access the server if you are on the same network as it. This is to help against intruders accessing your file system. This will soon change so that anyone can access the server as long as they have the IP address but they will be restricted to a user defined root directory, from which they will not be able to escape (mwhahaha). After testing the first build of the webserver it was noticed that some of the files had large chunks missing from them.

The standard way to send data over an estabished socket connection is through the send() function
ssize_t send(int sockfd, const void *buf, size_t len, int flags);
When the server was first constructed I assumed that this function would send all the data you tell it to, however it does not. After much research it was figured out how to force the code to send all the data. The third version of the protocol finally worked and we are now able to send the required data flawlessly. The program currently uses the unix only fork() function and will eventually need to be rewritten for the windows build. Until then more plugins will be develped

Graph Support For Plugins

This week saw the introduction of a javascript graphing library for Olympus.  The Raphaël Javascript Library is a new open source graphing library that is gaining popularity due to its simplicity.  After trying out a few examples ourselves, we implemented the library as a tool for plugin developers to quickly generate graphs of all kinds.  It is the perfect tool for presenting data, with a simple syntax for developers and clean, colorful and interactive graphs for end users.  We hope that by incorporating such a library in the Olympus Server Management framework we open up creative possibilities for developers and server admins to solve their server management needs.

Writing Our Own Webserver

In order to serve the live data from the remote computers to the client computer we needed to write at least part of our own web server to handle the AJAX requests for plug-in data. After writing a few dummy scripts, checking what out kind of input we will need to handle, it became very simple to handle and return data to the client computer. I wrote a C/C++ backed to handle the Ajax requests and send data back to the client computer. A port in bound on the computer then the program waits for a client to connect to it, initiating the AJAX query. After the web browser sends data to the server it is parsed and the actual request is separated from the meta-data. After the request is parsed, the server looks up the information and sends it back to the client in whatever format the data is needed in, not just HTML.

Web UI Hotkey(shortcut key) Support

In the pursuit of a feature rich Web UI, we looked to the interfaces of popular websites.  As web applications have become more mature, they have acquired many desktop experience-like properties that we find desirable in our application.    One feature in particular that has gained popularity in recent years is the ability to use shortcut keys on a webpage.  This is different from being able to use shortcut keys in the browser, as this is adding the ability to use shortcuts to interact with page elements.

Using its modern web technology stack, Olympus now supports basic Hotkeys.  Currently plugin windows are able to be opened using Shift+plugin_number where plugin_number is a number key, and they are able to be closed using Ctrl+Shift+plugin_number.  Hotkey support is being expanded for more functions within the app, so check back soon for updates.

Using jQuery, CSS3, HTML5 to create a usable frontend

Over the past two weeks much progress has taken place to the Frontend implementation to Olympus.  It was decided that the Web UI should be built with the future in mind.  Therefore, the project makes heavy use of modern web technologies – namely CSS3, HTML5 and the jQuery Javascript library.  CSS3 has been used generously in the details and styling of the Web UI.  HTML5 has not had as much emphasis, however the decision to use it has opened doors to increase the functionality of the project.  The open source jQuery library will be playing a major role in the project.  Aside from simplifying common tasks (ajax calls), the library allows for rapid development of functional UI.  It is the driving force behind the plugin ‘windows’ that appear within the interface as well as other details.  Thanks to the use of a modern web technology stack some possible future features for the Olympus Web UI include drag-and-drop functionality, hotkey/shortcut integration, offline-storage and the inclusion of geolocation into plugins.  Check back soon for progress.

Frontend Implementation

It was decided that the Web UI to Olympus should take the form of browser-accessed web pages.  At its core, each page uses ajax to serve the user content.  A user accesses a basic page from the root server and then live data is fed to the page through a series of ajax calls.  This cuts down on the need for a full-blown web server while at the same time providing the capability to serve real-time data.  A dashboard page loads a list of maintained remote servers.  The user can then click on a maintained server to see all of its plugins running in real-time, and if desired focus in on a particular plugin.