Tech Talk
One badass query to fix a big glitch
During development of OpenSim version 0.8.1 a bug was introduced in the code where simulators running on Windows would format xml statements wrong so they became unreadable on simulators running on mono with Linux. The result being that content created on a Windows based grid would not rezz when transferred to a Linux based grid. The odd thing being that the same content will rezz when transferred to an OS X based grid with the simulators running on mono.
The fix was in the OpenSim 0.8.2 development branch a few weeks ago to prevent new content being created with the wrong xml formatting, but the damage was already done as much content had both been created and re-uploaded after the OSGrid crash on simulators running Windows. Because of the very nature of Hypergrid content is being moved around, and the corrupted items are being spread between grids contaminating their asset databases.
To combat the spreading a Perl script was developed and released yesterday, and while this worked for MySQL based asset servers, it had to be reworked for PostgreSQL. – Not so much because the script was compatible by just changing a line of code, but because the way assets is stored on the PostgreSQL version of the OpenSim asset server.
The work-around was to take advantage of capabilities in PostgreSQL, and a badass query was developed that would fix the broken xml directly in the database without having to run external scripts. The query can be run directly in the psql console or a front-end like Navicat.
UPDATE assets SET data = DECODE(regexp_replace(ENCODE(data, 'escape'),'((xmlns:){2,})+', '(xmlns:)' , 'g'),'escape') WHERE "assetType" = '6';
There is a lot going on in this query:
It selects the asset records of type 6, sequentially goes through them by first encoding the content from binary format to textual UTF-8 format, then with the regexp_replace function seeks out the wrong formatting replacing it with the correct format. It then proceeds to decode the textual format back to binary, before it updates the record with the new content, and moves on to the next.
On our database server it looped through about 18500 records in 78 seconds which is about 237 records per second.
Be advised that before you try this at home, be sure to have proper backups of your data in case anything goes wrong and do the testing on copies of your data before you let the query loose.
Since native MySQL does not support the regexp_replace function, the above query cannot be used for the majority of OpenSim grid and standalone owners. It is possible to install a user developed function that has regex capabilities.
Visual Studio Code to the rescue
It is not often we recommend a Microsoft product, but when it comes to editing and working with the OpenSim source code, the beta version of Microsoft Visual Studio Code came to the rescue.
Having an editor on OS X that is .NET aware made an improvement over UltraEdit, and it is very fast too. The side by side editing mode on a big screen comes handy, as is having access to full folder hierarchies on the left hand sidebar for quick navigation.
With mono installed one can also compile and debug the code right from the editor.
You can get the beta, which is also available in a 32-bit Linux version, and more information from here.
Kokua Mac test viewer available
We have helped build a test version of the Kokua viewer for Mac.
It has been built with the Clang compiler in Xcode 6.4 and SDK 10.9, and is based on Linden Lab's 3.8.1 code base with the OpenSim additions and general improvements in the Kokua viewer.
This version also has a couple OpenSim related fixes in addition to all the bug fixes and improvements that is included in the Linden code (see the release notes for the SL viewer for details.)
It has been tested on all the major OpenSim grids both with direct login and hypergrid teleports in addition to SecondLife. You need OS X 10.8 or higher to run this version of the viewer.
This build is a major step forward for the Mac viewer since building it with Clang and not old GCC viewer that Apple discontinued support for in OS X 10.8 lays the foundation for building a 64-bit version of the viewer with the modern frameworks that Apple provides.
It can be downloaded via this blog post. Please report any issues via the Kokua ticketing system.
UPDATE: It is also possible to get the latest builds from Bitbucket while SourceForge is unavailable.