Life of a Programmer Geek

Saturday, December 25, 2010

Merry Christmas World!

Today I released a lot of my code as open source here. Enjoy!

Wednesday, December 8, 2010

Monday, December 6, 2010

Supposedly the Semantic Web contains lots of information, but what information? What is in there? Show me the money! I want to ask Wikipedia "What are all universities in the world and their student populations?" and I thought, what better test query for the Semantic Web?

I began my search by looking at various example queries through the DBPedia faceted browser:

[links from this page]

That's pretty cool, I can manage to list all universities, but how to display their student populations? I see no way of doing this with this faceted browser tool, which is really a faceted search tool, because all it allows one to do is filter the listing based on various parameters, not to specify which parameters are shown.

I think we'll need to learn some SPARQL to get at the student populations. I came across a nice SPARQL tutorial from XML.com and began following it. It seems like SNORQL is a widely used query interface for DBPedia. The SNORQL link from the DBPedia Applications page presents an example query executed on the DBPedia knowledge base: "List all names, birth dates and death dates for all people born in Berlin before 1900."

PREFIX dbo: <http://dbpedia.org/ontology/>

SELECT ?name ?birth ?death ?person WHERE {
?person dbpedia2:birthPlace <http://dbpedia.org/resource/Berlin> .
?person dbo:birthDate ?birth .
?person foaf:name ?name .
?person dbo:deathDate ?death
FILTER (?birth < "1900-01-01"^^xsd:date) .
}

Click here to see the result in the SNORQL Query Browser (GitHub page), a project by Richard Cyganaik for the D2R server project. Notice how nowhere in the query is "Person" specified - that is assumed given the fact that the resource has a "birth date".

I discovered something I had been wondering how to do: a query listing all the outgoing edges from a given resource, in other words listing all RDF (subject-predicate-object) triples which have a given resource as the subject. Here's a query which lists all the triples with Konraz Zuse (a listing in the previous result) as the subject:

SELECT ?property ?hasValue
WHERE {
{ <http://dbpedia.org/resource/Konrad_Zuse> ?property ?hasValue }
}

See the results by clicking here. Here are some triples returned:

property	hasValue
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://xmlns.com/foaf/0.1/Person
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/Scientist110560637
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/ComputerPioneers
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://www.w3.org/2002/07/owl#Thing
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/ComputerDesigners
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/ProgrammingLanguageDesigners
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/Werner-von-Siemens-RingLaureates
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/GermanInventors
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/ComputerHardwareEngineers
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/GermanCivilEngineers
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/class/yago/GermanComputerScientists
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/ontology/Person
http://www.w3.org/1999/02/22-rdf-syntax-ns#type	http://dbpedia.org/ontology/Scientist
http://www.w3.org/2002/07/owl#sameAs	http://umbel.org/umbel/ne/wikipedia/Konrad_Zuse
http://www.w3.org/2002/07/owl#sameAs	http://www4.wiwiss.fu-berlin.de/dblp/resource/person/120373
http://www.w3.org/2002/07/owl#sameAs	http://rdf.freebase.com/ns/guid.9202a8c04000641f8000000000022414
http://www.w3.org/2000/01/rdf-schema#comment	Konrad Ernst Otto Zuse war ein deutscher Bauingenieur, Erfinder und Unternehmer. Mit seiner Entwicklung des Z3 im Jahre 1941 baute er den ersten universellen Computer der Welt.
http://www.w3.org/2000/01/rdf-schema#comment	コンラート・ツーゼ（Konrad Zuse、1910年6月22日 - 1995年12月18日）は、ドイツの技術者である。彼の最も重要な業績は、1941年に世界初の完全動作するプログラム制御式コンピュータ Zuse Z3 を完成させたことである（プログラムはテープに格納）。1998年、Z3 はチューリング完全であることが証明された。何が世界初のコンピュータかという問題はコンピュータの定義に依存するが、Z3 は後のマシンと比較したときに汎用性に問題がある。ツーゼは高級プログラミング言語プランカルキュールを1945年に設計したが、これは理論的な部分での業績であり、彼の生きている間には実装されることもなく後のプログラミング言語にも直接的な影響を与えることはなかった。技術的な業績だけでなく、ツーゼは1946年に世界初のコンピュータ企業を設立した。この会社は世界初の商用コンピュータZ4を開発し、1950年にチューリッヒ工科大学にリースしている。第二次世界大戦の影響で、ツーゼの業績の大部分はイギリスやアメリカ合衆国では気づかれなかった。アメリカの企業で彼の影響が見られたのは 1946年にIBMがツーゼに特許使用許諾を得たのが最初である。1960年代後半になると、ツーゼは計算する宇宙（計算によって成り立つ宇宙）の概念を提唱した。 Z4とZ3の複製品がミュンヘンのドイツ博物館にある。ベルリンの Deutsches Technikmuseum Berlin はコンラート・ツーゼおよび彼の作品に関する特別展示をしている。再現されたZ1を含む12台の彼のマシン、オリジナルの文書、いくつかのツーゼの描いた絵などが展示されている。
http://www.w3.org/2000/01/rdf-schema#comment	Konrad Zuse var en tysk pioneer innenfor informatikk. Hans største bragd var konstruksjonen av den første funksjonelle datamaskin med programmer lagret på tape, kalt Z3, i 1941.
http://www.w3.org/2000/01/rdf-schema#comment	Конрад Цузе — немецкий инженер, пионер компьютеростроения. Наиболее известен как создатель первого действительно работающего программируемого компьютера и первого языка программирования высокого уровня.
http://www.w3.org/2000/01/rdf-schema#comment	Konrad Zuse was a German engineer and computer pioneer who collaborated with the German government during World War 2, which helped finance his projects. His greatest achievement was the world's first functional program-controlled Turing-complete computer, the Z3, in 1941 (the program was stored on a punched tape). He received the Werner-von-Siemens-Ring in 1964 for the Z3.

Wow. We get to know all the "kinds of person" he is known to be - a German inventor, scientist and computer pioneer - and summaries in many languages, and many more results not shown. Thats pretty impressive.

I played with the query a bit and remembered seeing the "DISTINCT" keyword in some SPARQL queries. Here's a variation on the above query that gives you a list of all unique property types which are applied to Konrad Zuse:

SELECT DISTINCT ?property
WHERE {
{ <http://dbpedia.org/resource/Konrad_Zuse> ?property ?hasValue }
}

Amazingly, the example queries from the XML.com tutorial actually execute through the DBPedia Snorql instance!

Very nice! Now we have some example queries for listing of some property values, a key ingredient in our "What are all universities in the world and their student populations?" puzzle. I see that the queries work, but how do I find out what vocabulary I can use? Google "DBPedia Ontology" and you're there in a few clicks. The DBPedia Ontology page contains many useful links. Here is the DBPedia ontology (class hierarchy) in text form. From that page there is a link to the University ontology class, which in turn lists all of its properties, including "numberOfStudents".

How can we use these things in a SPARQL query? We need URIs, not unqualified strings like "numberOfStudents". I noticed that in the result of a previous query, [Konraz Zuse, rdf:type dbpedia:ontology/Person] was a triple. If you expand the object into its full URI (PREFIX dbpedia: ) you get http://dbpedia.org/ontology/Person, which when accessed in a browser gives you a Linked Data interface to DBPedia. Just to see, I replaced Person with University, and sure enough, here is the Linked Data description of the DBPedia University class:
http://dbpedia.org/ontology/University, which reveals that University is the domain of the owl:Property http://dbpedia.org/ontology/numberOfStudents, which we can use in our query.

Here's the query that returns a listing of 10 universities from DBPedia:

PREFIX dbo: <http://dbpedia.org/ontology/>
SELECT ?university WHERE {
?university rdf:type dbo:University.
} LIMIT 10

So, Wikipedia, what are all universities in the world and their student populations?

...translates to...

PREFIX dbo:  SELECT ?name ?students WHERE {      ?university rdf:type dbo:University.      ?university foaf:name ?name.      ?university dbo:numberOfStudents ?students } ORDER BY DESC(?students) LIMIT 50

See the results here! Here are the top 50, formatted from RDF into HTML via the XSLT transformation provided with Snorql:

name	students
Indira Gandhi National Open University	3000000
California Community Colleges System	2900000
The Open University of China	2700000
Church Educational System	1200000
Florida College System	800000
Universitas Terbuka	580458
အဝေးသင် တက္ကသိုလ် (ရန်ကုန်)	560000
มหาวิทยาลัยรามคำแหง	525000
City University of New York	483000
The University System of Ohio	478367
State University of New York	438361
State University of New York	438361
Bangladesh Open University	432767
বাংলাদেশ উন্মূক্ত বিশ্ববিদ্যালয়	432767
California State University	417112
Chicago Public Schools	407955
Yashwantrao Chavan Maharashtra University	400000
Community College of the Air Force	351715
Centre national d'enseignement à distance	350000
University of Buenos Aires	308594
Universidad de Buenos Aires	308594
National Autonomous University of Mexico	305969
Universidad Nacional Autónoma de México	305969
Oklahoma State System of Higher Education	236000
University of Delhi	220000
DU	220000
दिल्ली विश्वविद्यालय	220000
Cairo University	200000
Gāmaʿat al-Qāhirah	200000
جامعة القاهرة	200000
University of Guadalajara	195071
University of North Carolina	183000
Korea National Open University	182859
Universidad Bolivariana de Venezuela	180000
Universidad Nacional de Educación a Distancia	180000
National University for Distance Education	180000
The Open University	168850
Universidad Autónoma de Santo Domingo	167533
UASD	167533
Sharks	161668
Miami Dade College	161668
Studium Urbis	147000
Sapienza – Università di Roma	147000
Modern University for the Humanities	140000
University of Bikaner	140000
University of London	135090
Universitas Londiniensis	135090
Universidade Norte do Paraná	130000
Norte do Paraná University	130000
Universidad Autónoma de Nuevo León	129341

Here's a version that gives you (lat,long) coordinates too.

Mission accomplished. It looks like the Semantic Web is quite promising after all. I look forward to seeing a user interface which would allow me to construct that query and view the results visually within one browser window - including all vocabulary researching.

What an incredible time to be alive - the collective body of human common knowledge is finally available for machines to process. I wonder what will come next.

Monday, November 29, 2010

Hello!

Greetings! This blog is a collection of guides and technical notes for doing various things, mostly in Ubuntu Linux. Here is a list the entries I find myself going back to often:

If you appreciate this site and want to support it, go for it!

Ubuntu
Ubuntu Introduction
Installation scripts

Editors:
Installing Eclipse
Emacs (and SLIME with Clojure)

Emacs Keystroke Reference
Vim

Flex in Gedit

Java:
Installing Glassfish

Installing Jetty

Deploying a WAR file in Jetty
Getting Started with Java Persistence API
R and Java

Getting started with Grails

SQL:
Day to day SQL reference

Flex:

Learning Flex
Flexbuilder in Linux
Flex, WebServices, Glassfish, Eclipse
Using the Open Source Flex SDK

Clojure:
Learning Clojure (and Emacs with SLIME)
MVC in Clojure
OpenGL in Clojure
3D Grapher in Clojure (using OpenGL)

LaTeX:
Getting Started with LaTeX
Pseudocode
LaTeX in VIM
LaTeX in Emacs

Graphics:
Installing Nvidia Drivers
OpenGL in Eclipse
OpenGL Cylinders and Spheres with inside test
Getting Started with CUDA in Ubuntu
CUDA in VIM
CUDA in Eclipse

Wordpress:
Installing Wordpress
Set up Email notifications

Unix:
Recursive word count

Text search across files
String replacement across many files
batch rename
delete all but a given type of file
remove all .svn directories
Redirect all output to a black hole

Utilities:
zipping and unzipping
using SVN (version control) command line client
Set the desktop background from the command line
Setting Up Synergy (cross-computer keyboard and mouse sharing)
NVidia Dual Monitor setup
Turn off the system beep and ALL system sounds
Code to HTML

Projects:

Musical Instrument using Processing and PureData

Frequency Modulation Pad with PureData and Processing

Simple Synthesizer using JACK and QT

Keyboard as Guitar Fretboard to Midi using JACK and QT

3D Grapher using Clojure and OpenGL

Tiny Lisp Parser and Abstract Syntax Tree Visualizer in Java

All code and text on this blog is in the public domain, free to use and modify with no restrictions whatsoever.

Enjoy!

--Curran

Tuesday, November 23, 2010

Design Axioms

Axioms for Human Computer Interaction Design

An executive summary of the excellent article by Juhan Sonin found at
http://www.mit.edu/~juhan/design_axioms.html

Let data scream (screen real estate: 85% data, 15% UI)
Always use real data
Prototype like crazy (fast development iterations)
Address layout, color, and interaction design from the start
Allow users to bitch about your service quickly (1-click feedback)
Dogfood your services
Ask for forgiveness rather than permission (just do)
Get continual feedback in the domain vernacular
Use grid hierarchies and basic information layouts
When in doubt, bang left
Pay attention to good typography practice
Use less than 5 type treatments of only 1 type face
Use better words and less of them
Color carefully
The document should be center stage, not the paint
What interface? Great interfaces disappear and have low cognitive overhead
People should be engaging directly with the content (e.g. iPhone photo app)
Manipulate the data, not the interface
Over time as you use a service or product, the interface melts into the background.
Design first for the repeat user (jedi), then novice, then the infrequent/inexperienced user.
Experience breeds familiarity, and in interface design familiarity promotes usability.
Cognitive heat sink: I get it, feels right, go here... instead of oh yeah, I remember where that was
Use keyboard shortcuts to supplement intuitive actions (enabling ninja speed)
Design every second of the experience
Performance ("snappiness") counts.
Products thrive (or wither) based on the users’ experience

Friday, August 27, 2010

Editing OpenStreetMap

Mapping a region and adding it to OpenStreetMap is a gratifying experience.

Today I thought I'd try out the OSMTracker Android app. I used it to record GPS traces of an unmapped area in Paxton, MA - walking trails near a small lake called Turkey Hill Pond. I rode my bike through the trails while the app was running. At the end I had these "GPX" files which I opened up in JOSM and used as the basis for adding new paths.

I noticed that the entire lake was missing so I installed the wms plugin for JOSM and used the Yahoo satellite images to trace out the lake. Tagging was straightforward once I found the tag definition page. I created an OSM account and uploaded the data, and within a few hours the image tiles were updated.

Here is a before and after screen capture of the OSM map:

How cool! Its a great feeling to know I've contributed something, and that anyone who looks at this region in OSM in the future will see these trails.

Next I'd like to map the Lowell state forest - there is no good map for all those trails.

Tuesday, August 24, 2010

Flex Builder in Linux

Here is how to get Flexbuilder working in Linux.

Set up the Flex 3.5 SDK:
cd ~/opt
mkdir flex_sdk_3.5
cd flex_sdk_3.5
wget http://fpdownload.adobe.com/pub/flex/sdk/builds/flex3/flex_sdk_3.5.0.12683.zip
unzip flex_sdk_3.5.0.12683.zip
rm flex_sdk_3.5.0.12683.zip
echo "PATH=\$PATH:\$HOME/opt/flex_sdk_3.5/bin" >> ~/.bashrc

Install Flexbuilder
cd ~/opt
mkdir flexbuilder
cd flexbuilder
wget http://download.macromedia.com/pub/labs/flex/flexbuilder_linux/flexbuilder_linux_install_a5_112409.bin
chmod +x flexbuilder_linux_install_a5_112409.bin
./flexbuilder_linux_install_a5_112409.bin

Now open Eclipse and add the Flex 3.5 SDK in Properties -> Flex Compiler -> Configure Flex SDKs -> add -> choose the folder ~/opt/flex_sdk_3.5 -> ok -> choose to use the "Flex 3.5" SDK.

Theoretically that should do it, but at the time I did this (8/24/10) some additional steps were required. First I got the error

java.lang.IllegalArgumentException: "The attribute value type is com.adobe.flexbuilder.project.compiler.internal.ProblemManager and expected is one of java.lang.String, Boolean, Integer"

which is a result of this bug. To fix this, follow the instructions here. The instructions were not clear on this point: you need to copy the class file into the inside of the jar file.

The next error I had trouble with was the following:

configuration variable 'target-player' must only be set once

This persisted even after I went into Properties -> Flex Compiler and unchecked the box that says "Require Flash Player version". It turns out that the player version is defined in the SDK in

flex_sdk_3.5/frameworks/flex-config.xml

I edited that file and commented out the line with the required player version. Even after this the error is still there for me... until I applied the patch described here. Now FlexBuilder in Linux is working! Woo hoo!

Monday, August 23, 2010

RhythmSample

Here is a small project I did which lets you play several samples at several frequencies - quarter notes, eighth notes and sixteenth notes. The GUI is written in Java using Processing as a library. The audio generation is done using PureData, driven by messages sent from the Java GUI over a socket.

Here is the code. To run it, first open the PureData patch (you can use the startPd.sh script for this or do it manually), then run the Java program. The Java code communicates to the PureData patch over a socket.

Here is how it sounds when played:

RhythmSample-sound by curran

Thursday, June 17, 2010

Slick Grails Error Reporting

Here's a slick way of nicely reporting errors using Grails' built-in validation machinery:

package adminprototype
class AdminController {
  def index = { }
  def createDatabaseConnection = {
    render Utils.saveAndReport(
      new DatabaseConnection(params))
  }
}

package adminprototype
class Utils {
  static String saveAndReport(Object o){
    return o.save()?"success":reportErrors(o)
  }
  static String reportErrors(Object o){
    def error = o.errors.getFieldError()
    String field = error.getField().toString()
    field = field.replaceAll("\\B([A-Z])", " \$1")
    field = field.toLowerCase()
    String bogus = error.getRejectedValue()
    return "The "+field+" cannot be "+(bogus?bogus:"blank")+"."
  }
}

Now if I leave out the field "databaseName" in a request like this:
http://localhost:8080/AdminPrototype/admin/createDatabaseConnection?port=1234
I get a really nice error message with the camel case field name converted into a space delimited name:
The database name cannot be blank.

Wednesday, June 16, 2010

Installing Jetty

Unfortunately the Jetty Ubuntu package doesn't include what is needed for JNDI, namely the "jetty-plus" stuff. Here's how to install Jetty manually:

mkdir ~/opt

cd ~/opt

wget http://download.eclipse.org/jetty/7.1.4.v20100610/dist/jetty-distribution-7.1.4.v20100610.zip

unzip jetty-distribution-7.1.4.v20100610.zip

mv jetty-distribution-7.1.4.v20100610 jetty

cd jetty

echo "

export JETTY_HOME=`pwd`

export PATH=\$PATH:`pwd`/bin" >> ~/.bashrc

restart your terminal

One way to start Jetty is:

java -jar start.jar

Now go to http://localhost:8080/ to test it.

Ctrl+c will stop this.

Jetty comes with a command line utility for starting and stopping Jetty as a service, here's how to use it:

jetty.sh start

jetty.sh stop

Links

Jetty Download Page

Running Jetty

Tuesday, June 15, 2010

Deploying a WAR in Jetty

Here's how to deploy a WAR file in Jetty using the Ubuntu Jetty package:

Install Jetty:

sudo apt-get install jetty -y

edit /etc/default/jetty so the 4th line reads

NO_START=0

To start Jetty:

sudo /etc/init.d/jetty start

go to http://localhost:8080/

To stop Jetty:

sudo /etc/init.d/jetty stop

To list all commands:

/etc/init.d/jetty6 help

To deploy a WAR file, copy it into

/usr/share/jetty/webapps

and restart the server. The root URL of the web app will be http://localhost:8080/warfilename/ where warfilename is the name of your war file.

Links:

Newbie Guide to Jetty

Jetty Debian package documentation

JNDI in Jetty

Tuesday, May 25, 2010

Unix Magic: text search across files

This command will find all occurrences of "table.sqlTable" across all files within the current directory and all subdirectories recursively (-r) and will give you line numbers (-n)

grep -r -n "table.sqlTable" ./*

oh yeah.

Sunday, May 16, 2010

Rediscovering Emacs

I get the idea that Emacs is all about power and efficiency, but somehow I fell out of using it because I would forget all those darn keystrokes and I had to USE the text editor RIGHT AWAY to get stuff done.

But now my curiosity is again piqued. Is it really possible to do all code editing operations while keeping my hands in the standard position - no arrow keys and no mouse? That would be sweet, and worth the learning time.

So here goes - a comprehensive documentary of relearning Emacs:

To open Emacs and have it stay in the terminal, use
emacs -nw

Basic Operations (C = ctrl, M = alt)
C-x C-c = close Emacs
C-x C-s = save file
C-x s ! = save all open files

C-x C-f = open file

Cursor Navigation
C-v = pg down
M-v = pg up
C-l = center on cursor text
C-l C-l = cursor text to top
C-l C-l C-l = cursor text to bottom
C-p = up arrow
C-n = down arrow
C-f = right arrow
C-b = left arrow
M-f = forward a word
M-b = back a word
C-a = home (cursor to beginning of line)
C-e = end (cursor to end of line)
M-a = forward a sentence
M-e = back a sentence
M-< = cursor to beginning of document M-> = cursor to end of document

Undo Redo

C-/ = Undo

C-/ [do stuff] C-/ = Redo (undoing previous undos)

Editing Text

? = backspace

C-d = delete

Essentials for Coding
C-j = intelligent newline (accounts for indentation)
Mark a region (for example M-> M-<>
C-M-\ = auto-indent region (auto-indent, auto-format)
M-g g = go to line
M-/ = autocomplete word

Search and Replace
C-s = search forward
C-r = search backward
C-M-s = regex search
M-% = incremental search and replace
M-x replace-string = global search and replace

Selecting, Copying, Pasting, and Marking
C-spacebar = set mark on cursor
Shift+(any cursor navigation) = set mark on cursor
C-x C-x = swap mark and cursor
C-u C-SPC = cycle through mark ring (previous 16 marks)
region = the text between the last set mark and the cursor
M-w = copy region
C-w = cut region
C-k = cut from cursor to end of line
C-y = paste
M-y = replace pasted text with previously copied text (scrolls through copy history)

Arguments
Arguments can be passed to any text editing or navigation operation:
C-u 123 = give '123' or whatever number you type as an argument to the operation you are about to do.
M-123 = the same thing. For example,
C-u 100 * = M-100 * = type 100 '*' characters
C-u 8 C-v = M-8 C-v = scroll up 8 lines
M-x z = repeat command

Cancel
C-g = cancel anything (scrap arguments, or cancel pending operation)

Macros
You can record a sequence of operations and then play it back repeatedly:
F3 = start recording macro (stuff you do is recorded)
F4 = stop recording macro
F4 = execute recorded macro

Buffers
When you have multiple files open, they are in "buffers".
C-x C-b = list buffers
C-x b = switch to buffer
C-x k = close buffer
C-x rightArrow = next buffer
C-x leftArrow = previous buffer

Windows
You can have many files open at once, either in their own "window" (screen partition) or in offscreen "buffers" (like tabs).
C-x 2 = split window vertically
C-x 3 = split window horizontally
C-x o (letter o)= move cursor between windows
C-x 0 (number zero) = close window
C-x } = expand window horizontally
C-x { = contract window horizontally
M-x z = repeat command (useful with C-x } and C-x {)

Variables
C-h v var RET = view variable value and documentation
M-x set-variable RET var RET value RET = set variable value

File Browsing and Manipulation
M-x dired = DIRectory EDitor (see manual)
...once in dired...
C-x C-q = make dired writable, so editing names as text renames actual files.
f = go into directory
^ = go up out of current directory
To delete files:
d = flag file for deletion
u = unflag file for deletion
x = delete files flagged for deletion

Diff
M-x ediff

Case
http://jamesthornton.com/emacs/node/emacs_244.html

Miscellaneous Linux Keyboard Tips
M-Tab = switch between windows
C-M-rightArrow = switch desktop right
C-M-leftArrow = switch desktop left

Version Control
see http://www.gnu.org/software/emacs/tour/ search for "Version Control"

Groovy in Emacs
Download groovy-mode:
sudo wget http://svn.groovy.codehaus.org/browse/~raw,r=HEAD/groovy/trunk/groovy/ide/emacs/groovy-mode.el -O /usr/share/emacs23/site-lisp/groovy-mode.el
You might need to replace 23 with your version number.

Put this in ~/.emacs:
;;; turn on syntax hilighting
(global-font-lock-mode 1)

;;; use groovy-mode when file ends in .groovy or has #!/bin/groovy at start
(autoload 'groovy-mode "groovy-mode" "Groovy editing mode." t)
(add-to-list 'auto-mode-alist '("\.groovy$" . groovy-mode))
(add-to-list 'interpreter-mode-alist '("groovy" . groovy-mode))

Links:
Groovy mode
Dired manual
Emacs Commands List
Guided tour of emacs

Sunday, April 18, 2010

Installing Protege

Here's how to install the Protege ontology editor in Ubuntu Linux:

Download Protege

wget http://protege.stanford.edu/download/protege/4.1/installanywhere/InstData/Linux/NoVM/install_protege_4.1.bin

Run the Installer

sh ./install_protege_4.1.bin

This will pop up a GUI installer.

I chose as my install directory "~/opt/Protege_4.1/", but anywhere will work.

Click through until you are done.

Run Protege

To run Protege, execute

/home/curran/opt/Protege_4.1/Protege

replacing /home/curran/opt with your Protege location.

Set up Protege as a command:

To set up "Protege" as a command, add the Protege directory to your PATH environment variable as follows:

echo "

export PATH=\$PATH:\$HOME/opt/Protege_4.1/" >> ~/.bashrc

replacing the details with the location you chose. Restart your terminal and now run the command (note upper case P)

Protege

Enjoy!

Saturday, April 10, 2010

Grails in Gedit

To install Grails plugin for Gedit:
cd ~/opt
git clone git://github.com/aeischeid/gedit-grails-bundle.git
cd gedit-grails-bundle/grails-gedit/
sudo ./install.sh
Restart Gedit and woo hoo!

Links:
Grails Gedit page on Grails site
gedit-grails-bundle in GitHub

Tuesday, March 30, 2010

Flex in Gedit

Man I really do like coding in simple environments - it's like walking barefoot on the beach. So, I figured, why not get acquainted with Flex not through sluggish autocompletions and feigned build magic, but rather by relying only on solid tooling, a simple text editor, and documentation.

The mxml is legible and colorful, while the actionscript demands a more focused eye and takes longer to digest. This is the default syntax highlighting of mxml in Gedit - as XML. There is no built-in support for mxml and actionscript.

Luckily, someone has created mxml and as3 language definitions compatible with Gedit. Here's how to install them:
cd /usr/share/gtksourceview-2.0/language-specs
sudo wget http://conic.se/static/blog/code/Langfiles/actionscript.lang
sudo wget http://conic.se/static/blog/code/Langfiles/MXML.lang
sudo update-mime-database /usr/share/mime
restart Gedit, and voila!

Balance. Much improved legibility.

In Windows
The parallel to
/usr/share/gtksourceview-2.0/language-specs
in a Windows Gedit installation is
C:\Program Files\gedit\share\gtksourceview-2.0\language-specs

Put the MXML.lang file there and restart Gedit.

A Fix for fx:Script and mx:Script
Initially, the MXML.lang file does not handle the new fx:Script tag, which makes it impossible to use with Flex 4 code.

A partial solution is to replace all occurences of "mx:" with "fx:" in MXML.lang. This will make fx:Script work, but will break mx:Script.

I was able to get Gedit to recognize both the mx:Script and fx:Script tags by editing the MXML.lang file:

sudo gedit /usr/share/gtksourceview-2.0/language-specs/MXML.lang

and adding the following code

<context id="actionscriptfx">
   <start>&lt;fx:Script&gt;</start>
   <end>&lt;/fx:Script&gt;</end>
  
   <include>
   <context sub-pattern="0" where="start" style-ref="element-name"/>
   <context sub-pattern="0" where="end" style-ref="element-name"/>
   <context ref="element-name"/>
   <context id="actionscript-codefx" extend-parent="false">
       <start>&lt;!\[CDATA\[</start>
       <end>\]\]&gt;</end>
       <include>
           <context sub-pattern="0" where="start" style-ref="cdata-delim"/>
           <context sub-pattern="0" where="end" style-ref="cdata-delim"/>
           <context ref="actionscript:actionscript"/>
       </include>
   </context>
   </include>
</context>
...
<context id="mxml">
<include>
<context ref="comment"/>
<context ref="doctype"/>
<context ref="actionscript"/>
<context ref="actionscriptfx"/>
<context ref="cdata"/>
<context ref="prolog"/>
<context ref="processing-instruction"/>
<context ref="start-tag"/>
<context ref="end-tag"/>
<context ref="entity"/>
<context ref="character-reference"/>
<context ref="unallowed-chars"/>
<context ref="close-tag-outside-tag"/>
</include>
</context>

Then run

sudo update-mime-database /usr/share/mime

and restart Gedit.

Update: After all that work I found out there is an up to date version here:
http://github.com/julien/gedit_flex
will update this post once I try it out.

Saturday, March 27, 2010

Using the Open Source Flex SDK

I'd like to do some Flex development without Flexbuilder. Here's how I got the open source SDK working in Ubuntu Linux:

Installing:

Download the SDK from the download page. Unzip its contents to ~/opt/flex as follows:

cd ~/Downloads

unzip flex_sdk_4.zip -d ~/opt/flex

Add the Flex bin directory to your PATH:

echo "PATH=\$PATH:\$HOME/opt/flex/bin" >> ~/.bashrc

restart your terminal. Now you should have the mxmlc command and other Flex command line tools available:

Writing Hello World:

<html>
<head>
<style type="text/css">
<!--
body {
height: 100%;
width: 100%;
margin: 0;
}
-->
</style>
</head>
<body>
<object width="550" height="400">
<param name="movie" value="hello.swf">
<embed src="hello.swf" width="100%" height="100%">
</embed>
</object>
</body>
</html>

Here we have a typical HTML page for displaying a .swf file. And below is a "Hello World" mxml file.

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
   <mx:Label text="Hello World!"/>
</mx:Application>

You can download, compile, and run these files as follows:
wget http://www.curransoft.com/code/hello/hello.mxml http://www.curransoft.com/code/hello/hello.html
mxmlc hello.mxml
firefox hello.html

There you go!

Friday, March 26, 2010

Hilarious

I find this set of instructions very funny, from the kind folks at ITIS:

3. Installation instructions.

 Windows Instructions

   After downloading the ITIS MySQL zip file, that
   file "itisMySQL111908.zip" will now reside on your
   system as a Compressed (zipped) Folder.

   Use WinZip or some other software to extract the
   following 4 files from your downloaded ITIS zip file:
     installdb.bat createdb.sql ITIS.sql ReadmeMySQL.txt

   Use "My Computer" to go to where you have
   extracted those 4 files.

   On your Windows system, make sure that
   you can run the MySQL software as the
   MySQL user root on your PC.

   If you can't run MySQL as the MySQL user root
   on your PC then edit file "installdb.bat"
   and replace root with your chosen MySQL user name.

   Make sure that the MySQL Server is running on
   your PC prior to double clicking on the icon.

   Caution:  When you install this new ITIS database
   in MySQL on your PC, the SQL command to drop any
   preexisting database called "itis" is first executed
   before creating and loading the new "itis" database.

   To install the ITIS database onto MySQL on your PC,
   double click on the icon:
      installdb.bat

   Note: You will be prompted for your MySQL password after
   clicking on the icon.
   Be patient when waiting for the data to load into MySQL.
   The loading of the data may take 2 minutes or more.

 Linux Instructions

   On Linux, execute the following command:
     mysql -uroot -p < createdb.sql

   Note: You may use a different MySQL user name other than root.

 Mac OS X Instructions

   On Mac OS X, execute the following command:
     mysql -uroot -p < createdb.sql

   Note: Again, you may use a different MySQL user name other than root.

Everyday SQL

I have been using SQL from time to time with no formal database education, and find myself looking up the same stuff over and over, so I thought I'd collect solutions for such common things here for reference.

Setting up:
I'm using Ubuntu and MySQL. To install MySQL:
sudo apt-get install mysql-server mysql-client -y
This installer will prompt you for a root password.
To run the command line SQL client as the SQL root user:
mysql -u root -p
-u means user, -p means it will prompt for a password. Now you should be at an SQL terminal that looks like this:
mysql>
To list databases:
mysql> SHOW DATABASES;
To "use" a database (which is akin to "cd" in Unix), for example the "information_schema" database built into MySQL:
mysql> USE information_schema;
To list tables:
mysql> SHOW TABLES;

By the way, the convention for SQL is to use upper case, but it is case insensitive, so
mysql> show tables;
would work.

Creating a Database:
mysql> CREATE DATABASE test;
mysql> USE test;

Creating a table:
with column "foo" of type string with max length 5 and column "bar" of type int:
mysql> CREATE TABLE test (foo VARCHAR(5),bar INTEGER);

Inserting values (adding rows):
mysql> INSERT INTO test VALUES ("A",1);
mysql> INSERT INTO test VALUES ("B",2);
mysql> INSERT INTO test VALUES ("C",3);

Querying for everything:
mysql> SELECT * FROM test;
+------+------+
| foo | bar |
+------+------+
| A | 1 |
| B | 2 |
| C | 3 |
+------+------+
3 rows in set (0.00 sec)

Querying for a specific field:
backticks are used for quoting but are not necessary for column names without spaces.
SELECT `foo` FROM test;
+------+
| foo |
+------+
| A |
| B |
| C |
+------+
Querying with a filter:
mysql> SELECT * FROM test WHERE foo='B' or foo='C';
+------+------+
| foo | bar |
+------+------+
| B | 2 |
| C | 3 |
+------+------+

Deleting specific records:
mysql> DELETE FROM test WHERE foo='B' or foo='C';

Deleting a table:
mysql> DROP TABLE test;

Deleting a database
mysql> DROP DATABSE test;

Exiting:
mysql> exit
Bye

Reading a CSV file:
From a unix shell, I'll download a sample CSV file:
wget http://www.curransoft.com/data/iris.csv
Go into the SQL shell:
mysql -u root -p
Create an empty table with column names and types mirroring the CSV:
mysql> USE test;
mysql> CREATE TABLE `iris` (`SepalLength` float,`SepalWidth` float,`PetalLength` float,`PetalWidth` float,`Class` varchar(20));
Load in the CSV file with the following SQL:
LOAD DATA LOCAL INFILE 'iris.csv'
INTO TABLE iris
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(`SepalLength`,`SepalWidth`,`PetalLength`,`PetalWidth`,`Class`);

The delimiter is specified by FIELDS TERMINATED BY
The quote symbol is specified by ENCLOSED BY
The newline symbol is specified by LINES TERMINATED BY
The rest specifies which columns to map the CSV columns to.

Dumping a table to a SQL script file:
The mysqldump utility generates a SQL script for recreating a given table. It outputs to stdout, so its output must be redirected to a file. From a unix shell, here's how to output the iris table into iris.sql:
mysqldump test iris -u root -p > iris.sql

Loading a SQL script (backup file):
This will load the generated iris.sql file, which creates and populates the iris table on the currently used database.
mysql> source iris.sql;

Giving access to the outside:
To open your MySQL instance to the outside world, you first need to modify my.cnf:
sudo gedit /etc/mysql/my.cnf
such that
bind-address = your_ip
where your_ip is the "inet addr" field of the output of the command
ifconfig

The second thing you need to do is grant access to a specific user name and password for a specific computer:
mysql> GRANT ALL ON database_name.table_name to 'username'@'client_ip' identified by 'password';
for example:
mysql> GRANT ALL ON test.* to 'fred'@'192.168.45.32' identified by 'fred123';

Start/Stop/Restart the MySQL server:

sudo /etc/init.d/mysql start

sudo /etc/init.d/mysql stop

sudo /etc/init.d/mysql restart

Enjoy!

Thursday, March 25, 2010

Grails

I decided to try out Grails in Ubuntu. Here's what I did to get Grails up and running:

Installation:

Download the Grails binary from the Grails download page.

Unzip it into ~/opt, so everything will end up in ~/opt/grails-1.3.0 or whatever version you get. This directory will be GRAILS_HOME.

We will also need to set JAVA_HOME. You can see a list of installed JDKs (DO NOT use a JRE for JAVA_HOME) by executing

sudo update-alternatives --config javac

My machine shows only one: /usr/lib/jvm/java-6-openjdk/bin/javac, which I'll use as JAVA_HOME, minus the /bin/javac.

To set JAVA_HOME, GRAILS_HOME, and append the Grails bin directory to PATH, execute the following as a single command in a terminal (shift+insert to paste) with your correct paths substituted (this command appends the text to the end of .bashrc, which is executed every time a shell starts):

echo "

export JAVA_HOME=/usr/lib/jvm/java-6-openjdk

export GRAILS_HOME=\$HOME/opt/grails-1.3.0

export PATH=\$PATH:\$HOME/opt/grails-1.3.0/bin" >> ~/.bashrc

Restart the terminal.

Quick Start:

Now lets run through the quick start:

grails create-app takes the name of the project as an argument, for which it creates a project directory. I'm going to make a project in ~/grailsCode/book as follows:

cd ~/

mkdir grailsCode

cd grailsCode

grails create-app book

This creates the directory book with a bunch of stuff in it.

cd book

grails create-domain-class book

This creates a directory structure and groovy file for a "book" domain class. Edit this file:

gedit grails-app/domain/book/Book.groovy

such that it contains

package book

class Book {
String title
String author
static constraints = {
}
}

Now add some test data by editing this file

gedit grails-app/conf/BootStrap.groovy

such that it contains

import book.*
class BootStrap {

def init = { servletContext ->
// Create some test data
new Book(author:"Stephen King",title:"The Shining").save()
new Book(author:"James Patterson",title:"Along Came a Spider").save()
}
def destroy = {
}
}

"import book.*" is not in the original quick start. I think at some point in the past Grails did not use packages for its domain classes, and the quick start has not been updated? Now it does use packages, so the class name "Book" is not recognized without importing "book.Book" or "book.*".

Now create a controller:

grails create-controller book

Now edit the file

gedit grails-app/controllers/book/BookController.groovy

such that is contains

package book

class BookController {

def scaffold = Book
}

(yes, you must remove "def index = { }", otherwise it will not work)

Now run

grails run-app

You should eventually see "Server running. Browse to http://localhost:8080/book"

When I first ran this, I got the error Error executing script RunApp: : Error starting Sun's native2ascii:

This was because my JAVA_HOME was pointing to a JRE instead of a JDK. I changed it to point to a JDK (and updated the instructions above) and it worked!

So at http://localhost:8080/book I find a sweet autogenerated GUI for creating, removing, and listing Books!

There is so much magic happening I am a bit overwhelmed. What is responsible for that GUI? Where is the data stored? How can I change the home page display? Lots of questions, but at least I'm up and running.

Using MySQL:

A Grails project uses an in-memory HSQLDB database by default, which is wiped out every time you change the code or restart. I'd like to configure it to use a local MySQL instance, and to not delete the data on restart. Here's how to do this:

Create the database to use and grant access to a "grails" user:

mysql -u root -p

mysql> CREATE DATABASE book;

mysql> GRANT ALL ON book.* TO 'grails'@'localhost' IDENTIFIED BY 'g123';

Edit the file

gedit grails-app/conf/DataSource.groovy

such that it contains

...leave the rest of the file the same...
environments {
development {
dataSource {
pooled = true
dbCreate = "update"
url = "jdbc:mysql://localhost/book"
driverClassName = "com.mysql.jdbc.Driver"
username = "grails"
password = "g123"
}
}
...

Now we need the MySQL Connector jar, which we could go download and stick in the lib directory, but there is a better way. We can instead add the jar as a dependency using Grails' built-in dependency management system built on Apache Ivy using the Maven central repository as follows (the build system will download the jar for you - how sweet is that!):

Edit the file

gedit grails-app/conf/BuildConfig.groovy

uncomment the lines containing

mavenCentral()

and

runtime 'mysql:mysql-connector-java:5.1.5'

so you get

...
repositories {
grailsPlugins()
grailsHome()
mavenCentral()
}
dependencies {
runtime 'mysql:mysql-connector-java:5.1.5'
}
...

then run

grails run-app

It works! It took a while on the "Resolving dependencies..." step as it downloaded the MySQL connector jar. Now if you create a new book and stop the server, the new book will still be there the next time you start it (this behavior is specified by the dbCreate = "update" setting).

The last gotcha is to remove the automatic adding of books from the bootstrap file. As it stands, that code will persist new copies of those preset books upon every restart. Edit the file

gedit grails-app/conf/BootStrap.groovy

such that the "init" closure is empty:

def init = { servletContext ->
}

Now you'll have no more books automatically added.

Domain Object Relationships:

So far all we have seen is Grails magic - I still don't know how to actually write a webapp. Lets add tags to our books to get more familiar with Grails. This process is inspired by the Bookmarks example from the Definitive Guide to Grails by Graeme Rocher, the creator of Grails.

Lets create the Tag class:

grails create-domain-class tag

now to add its relationship to Book, edit the file

gedit grails-app/domain/book/Tag.groovy

such that it contains

package book
class Tag {
static belongsTo = Book
Book book
String name
static constraints = {}
}

The line

static belongsTo = Book

signals to the object relational mapping system of Grails (called GORM, built on Hibernate) that in the database, a Tag is owned by its corresponding Book. This means that when a Book is deleted (from the books database table), all its tags are also deleted (from the tags database table), and not the other way around.

Now we need to tell GORM that there is a one-to-many relationship between Books and Tags. Edit the Book class

gedit grails-app/domain/book/Book.groovy

such that it contains

package book
class Book {
static hasMany = [tags:Tag]
String title
String author
static constraints = {
}
}

Tag doesn't yet have a controller, create one with

grails create-controller tag

package book
class TagController {
def scaffold = Tag
}

now when you run

grails run-app

you will have a gui for CRUD operations on both Books and Tags, which has the correct delete behavior.

Customizing toString and adding validation constraints:

We can see from the listing of books in the tag selection GUI that a book is represented as "book.Book : $id". We can easily change this by overriding the toString method of the Book class as follows:

package book
class Book {
static hasMany = [tags:Tag]
String title
String author
static constraints = {
   title(blank:false)
   author(blank:false)
}
String toString(){"$title by $author"}
}

In Grails, quoted strings are in fact "GStrings" and support string interpolation - so you can access variables in scope and even write inline Groovy code using the $ escape character (use ${...code...} for more elaborate Groovy code)

I also added the constraints that the author and title fields cannot be blank. GORM automatically validates the instance using these constraints right before persisting to the database, in the call to save(). Lets add a constraint to the Tag class as well:

package book
class Tag {
static belongsTo = Book
Book book
String name
static constraints = {name(blank:false)}
}

Now, when we try to make a new Tag whose name is blank, an error with corresponding GUI results:

We can also see that our string interpolation worked correctly: "$title by $author" evaluated to "The Shining by Stephen King".

Enjoy!

Links:

Grails Installation instructions

Grails Quick-Start

Ubuntu page on environment variables

Grails dependency resolution

Life of a Programmer Geek

Saturday, December 25, 2010

Wednesday, December 8, 2010

Monday, December 6, 2010

Monday, November 29, 2010

Tuesday, November 23, 2010

Friday, August 27, 2010

Tuesday, August 24, 2010

Monday, August 23, 2010

Thursday, June 17, 2010

Wednesday, June 16, 2010

Tuesday, June 15, 2010

Tuesday, May 25, 2010

Sunday, May 16, 2010

Sunday, April 18, 2010

Saturday, April 10, 2010

Tuesday, March 30, 2010

Saturday, March 27, 2010

Friday, March 26, 2010

Thursday, March 25, 2010

Labels

About Me

Blog Archive