CVGM.net - Chiptune, Retro & Video Game Music

Hi,

I have spent some time working on a VM image for VirtualBox that will let users not too familiar with configuring a Virtual Machine, get started with hacking around on code within the Demovibes environment. The VM was built using VirtualBox (available to download for free at https://www.virtualbox.org/ ) and once downloaded/unpacked somewhere, follow the Import Machine options to install and copy the files to get it running. In the past, some VM imports have had no network due to various drivers etc. so I installed a Kubuntu environment (Kubuntu 10.10 64-bit edition) so that the VM can still boot to a desktop, and can still access a browser that will let you navigate the site locally. I will post a list of all the various settings and passwords needed for the VM later in this post.

The VM is set up to be a very simple instance of the running code, and by default it will look and behave very differently to CVGM. It's purpose is to give you the ability to tinker with a bit of code here and there, or just familiarise yourself with some of the superuser options within the system. It is expected that the user has some limited linux experience when using the VM. It may be possible to install the project onto a Windows VM, but it is not something that I wish to do personally.

To import the machine, unpack the 7z files somewhere and from within VirtualBox, go to Machine -> Add and browse to the DemovibesDev.vbox file. No errors should appear. Once imported, it will show in the list of Virtual Machines so you can configure it even further for your personal preferences. To launch it, just double click it.

Once the machine is imported and started, it will automatically boot to the KDE login prompt, where you can opt to log in or if you can access the VM from your host directly, you won't need to log in and run the extra KDE modules. I will give a brief pictorial step by step on how to get everything working and running, so that you can familiarize yourself with how it works. The VM uses the current SVN branch of code, so if this machine gets to be a bit old, you can run 'svn update' in the cvgm.net project folder, and the svn code will download the most recent changes. It will not allow you to post new code changes unless you are authorized to do so on the project page (located at http://code.google.com/p/demovibes-cvgm/ ). If you have any questions about the code, please use the regular forums on CVGM.net, or post bugs/issues directly on the project page. I'll post more complex upgrading procedures later if people need to know how to handle them, such as DB changes.

System Logins/Passwords/Etc.
The following is a series of common logins and passwords used by the VM. It is worth noting that this VM is intended purely as an educational resource. It is not intended for any sort of production use. Do so at your own risk!

Main Linux Account Login: cvgm Pass: demovibes (Super User password (su): demovibes)
MySQL Root Password: demovibes
Demovibes SuperUser: FishGuy876 Pass: password

Starting The System
When the VM boots, it will automatically start and launch Cherokee (the web server), along with the uwsgi event system and a few other bits and pieces in the background, such as Icecast2. Services such as the streamer and sockulf are not automatically started. The rest of this startup tutorial assumes that you are logging in and starting via the KDE desktop, but you can use your external SSH client if you configure your VirtualBox environmant to do so.

Click the KDE launcher (On the bottom-left where the windows start button normally is, it has a gear icon with a K in it), and select the Terminal program (Konsole). To save some time, you can switch to root mode by typing 'sudo su' and entering the superuser password above, otherwise you can do it for each tab, or run as a regular user once the system is properly confiured. If this is the first time that you have done any of these additional startup steps, type 'screen' (without the quotes) and press Enter, otherwise you can type 'screen -r' to resume an existing screen session. I have added a config to screen that shows some basic information about the VM so you can track what it is doing. At the bottom is the system load, and several screen pages, which you can use the keyboard shortcuts CTRL+A+P and CTRL+A+N to go to the Previous and Next screen pages. By default, screen will open on the sockulf page. Other pages that you will often come to use are demosauce, top and shell.

Screenshot of the Bash console with screen, on the top page:


The first service that we need to start is sockulf. This acts as a background engine between the site and the streamer. It handles any interaction with the song data, such as playing songs, uploading songs, scanning songs for length information, and a few other bits and pieces. For sockulf to work, demosauce must be compiled and working properly (which it is with this VM) and doesn't have to be running. Switch to the sockulf page. To start sockulf, you can enter the following commands:

cd /home/cvgm/cvgm.net/demovibes
source /home/cvgm/cvgm_virt/bin/activate

The top line changes the current directory to the demovibes code folder. The second line activates the Python Virtual Environment, which is a non-global environment useful for storing various Python and other system modules, without having to change any global ones. To learn more about how it works, read up on python-pip. Without activating the environment, the system will not function correctly at all.

I have also created a bash macro for the activation. After cd'ing to the directory, ust type 'source-activate' into the bash window. It should give no errors, and show (cvgm_virt) to the immediate left of the cursor to indicate it is active.

I also created a small scrit to handle sockulf, which will start the python files, and also restart them in the event there is some kind of crash or that it drops out. Just type in './start_sockulf.sh' and press Enter. If its able to start properly, you should see the following screen:



Time To Lay Down Some Serious Tunes
Once sockulf is up and running, the next step is to start the streamer. This step is relatively straight forward and will take care of itself even if problems arise during its operation. Like sockulf, a bash script is started to keep the service from stopping completely in the event there is a crash or a bad upload.

Switch to the demosauce page, and type the following commands to get it going:

cd /home/cvgm/cvgm.net/demosauce
./start_demosauce.sh

No special environment parameters are required to start the streamer. Once started, a message saying 'streamin'' will appear. the only other text that should appear here are song errors, crashes or other failures/warnings. Once started, you should immediately notice that the 'Now Playing' section of the site starts to count down to the next song in the queue, or a random request if the queue is empty.

Next Steps
For the basic demovibes setup, thats the meat & potatoes of getting your simple system up and running. You can always monitor the system load as you do various things on the site by going to the 'top' page, and seeing which tasks are doing what. With a larger database, such as cvgm, after a fresh reboot and start up, the uwsgi tasks and mysql ill often show a fairly high load as data is pulled from them. Depending on the data, most of it can be cached and so the next time its needed, the load on the system is not as high.

Making Changes To the Code
Now that everything is up and running, you want to start making some changes to the various files to see how they will work. If you plan on making any changes to the code, it is reccomended that you enable debugging mode. To do this, add the following line to cvgm.net/demovibes/settings_local.py (remember, you never want to edit the values in settings.py!) - 'DEBUG = True'. If you don't enable debugging mode, the site will not generate any info about the error (you may just get a 503 or 500 error message). When enabled, a debugging screen is shown that tells you about the error, and lets you track it a lot better.

When changes are made, they will not immediately become visible to the website. The uWSGI system needs to be restarted. This can be done on the command line using something like 'killall -9 uwsgi' and keep repeating until it tells you that no processes were found. Also, depending on what you are modifying code-wise, you may also want to restart sockulf, as it too will remember parts of the old code. For simple template and layout changes, it really isn't required.

The VM uses memcached for additional data caching for added speed, which is more apparent in a production server than this simple test machine. From time to time, the data in the cache needs to be flushed, either because its timed data (meaning it won't change until an elapsed amount of time has passed), or won't change until the data itself is altered somehow. To simply flush the entire cache, type the following into the command line: '/etc/init.d/memcached restart'

You can monitor the usage of memcached data within Demovibes. If you are a super-user only, you can go to the following url within the VM: http://localhost/demovibes/status/cache/ (It will generate a DivideZero error if there is no data inside of memcached yet).

The following is a screenshot from the CVGM memcached status page:


And once everything is working, you should see something similar by default:


Grab the current version of the Virtual Machine here: http://www.andykellett.com/files/DemovibesDev.7z (7z, 1.2Gb)

As always, if you have any questions about this VM or maintenance etc. feel free to post here and ask. Thanks, and enjoy!

FishGuy876

A couple of additions

Once you have started the streamer, launch Amarok and there should be a playlist entry in the right hand side already. Double click it, and it should connect to the stream and play.

Alternatively, install another player and connect to http://localhost:8000/mount.mp3

Icecast is set with all the default admin passwords (hackme)

Finish faster, stupid Microsoft Help updates. There is serious stuff to leech.

<doublepost> but I may as well use it and update thread with the info, that http://www.andykellett.com/files/DemovibesDev.7z is now only 1GB in size.

Hi,

Just a quick update to this post to explain how to move forward when a South database migration occurs. South is a really useful tool that monitors the relationship between the MySQL/SQLite/Whatever DB and the code, and creates special scripts that allow the movement of the data structure and data between them. You can also create scripts with south that migrate and process data between the different movements, so if a user would make a big DB change and added new fields, and removed old ones, the migration script will ensure the data goes to the correct new fields.

South migrations typically don't happen too often on CVGM, but they do from time to time and it's a good thing to know how to apply them correctly without breaking anything. Without a correct migration, you will expreience errors in the code such as fields not existing, as templates and other areas of the code may be looking for the new data.

Getting The Migration To Update
cd to the demovibes folder, which in the VM is the following:

cd /home/cvgm/cvgm.net/demovibes
source-activate

Now that the python environment is active, we will firstly sync the DB. This is also an important step, especially if some internal Django stuff has been happening.

python manage.py syncdb

This command should exit with no errors, signalling that the sync went OK. Now migrate the new changes:

python manage.py migrate

Again, this should follow through with no errors. Depending on what data needs to be changed, it could take anywhere from a few seconds to a few hours. Once this has done, restart the uWSGI instances and the new changes should take effect.