Anton Andreev's blog

The "Corpus Quality Assurance" tool and Annotation Transfer processing resource make it easy to compare the results of two pipelines. (when it works).

You won't find this tool in the "Tools" menu, but at bottom of the GATE interface when you click on a corpus.
In order to use Corpus Quality Assurance we need to two annotations sets so that we can compare them.

1. Create datastore

Create a corpus. Then right click it and use the "Save to datastore" option and point it to your datastore. Next populate the corpus and see how the documents do not appear as usually under the "Language resources" tab, they go to the datastore. Datastores are quite useful as they do not keep all the documents in memory. This way you can annotate a large number of documents before you encounter a problem with GATE or JAVA.


2. Create Annotation Set 1

Warning: Check which version of GATE are you using and what are the plug-ins loaded. GATE can load plug-ins from other GATE versions and instances. You might for example want to test against another version of GATE. But probably the second GATE will load the same list of plug-ins as the one you have just closed. Yes, the core of the GATE framework will be different, but because you load the same plug-ins, you will probably get the same results. So you might end up comparing the same results or wondering why the results are not one you expect. Now go to the File->Manage CREOLE Plugins and do the job of selecting your resources explicitly.

Add a new "Annotation Set Transfer" processing resource to the end of your pipe-line:

inputASName should be blank to use the default annotation set.
outoutASName will be our case "Annotation Set 1".
For more information see the documentation.

Now run the pipeline. You should find the new annotation set below the default one. If needed - collapse the default one to see the new one.

This new annotation set "Annotation Set 1" has been saved in the datastore.

2. Make changes
Replace plug-in versions or load another instance of GATE. Please check the plug-in versions of this new GATE instance.

3. Create Annotation Set 2

Open the datastore created in step 1(if it is not already available).
Open the corpus contained in that datastore (if it is not already available) by clicking the datastore, scrolling down until you find it, then right click and select "Load". It should now be available in its usual place under the "Language resources".
Check if "Annotation Set 1" is available.

Create "Annotation Set 2" by again adding "Annotation Set Transfer" processing resource and setting the correct parameters. Do not forget to remove "Document Reset PR" in the beginning of your pipeline or otherwise it will clear "Annotation Set 1".

Run the pipeline!

4.
Click on the corpus and then on the Corpus Quality Assurance at the bottom. Wait until it initializes, there is a progress bar on the right. this initiazlization is need so that the tool knows what are you annotations types and features so it can fill the lists on the right.

Select "Annotation Set 1" and "Annotation Set 2" as A and B in the first list.
Then select the annotations that you are interested in. The select the features by which you think two annotations are identical.
Select measures and then press the "Compare" button.

Click on "Document statistics". The click on a document where the "Only A", "Only B" or "Overlap" are greater then 0. This means we have a difference, click on the document and then click on the "Annotation Diff" button which is currently the second one on the right top of GATE.

You can test the count implementation of Ontotext in the OWLIM semantic database at the following address: ldsr.ontotext.com/sparql

In the form type the following query:


Please note that this functionality is only available in Big OWLIM 3.2.7 and above.

This post simply proves that GATE 5.1 NLP framework runs on OpenSolaris 2009.06 without any hassle. OpenSolaris uses bash as its default shell script language.

1. Download the GATE archive. Mine is "gate-5.1-build3431-BIN.zip".

2. Unzip the archive:
# unzip gate-5.1-build3431-BIN.zip

3. Install SUNWj6dev in order to have the Sun JDK 1.6 and not only the JRE.
# pkg install SUNWj6dev
You can learn more about the pkg - OpenSolaris Image Packaging System from here.

4. Issue the following commands in bash:
# JAVA_HOME=/usr/jdk/instances/jdk1.6.0
# echo $JAVA_HOME
Make sure that your JDK is really in /usr/jdk/instances/jdk1.6.0

5. Go to the bin folder of GATE and simply type:
# ./gate.sh

The following article will give you basic knowledge on how to move a GATE application from one GATE instance to another.

Each GATE application is stored as a .gapp file.
For example KIM's default pipeline is stored in "IE.gapp" file. Indeed a gapp file is an XML file.

1. Gapp file considerations

1.1 Best place to save a gapp file:
Usually this should be somewhere in the tree of a GATE/KIM deployment. When you save your gate application(also called pipeline) the resulting gapp file is generated with relative paths to your GATE/KIM installation.

1.2 File-systems paths:
Usually a path in a gapp file looks like that:

$relpath$../../../C:/Program%20Files/GATE-5.0/plugins/Ontology_Tools

It is not recommended to use absolute paths, but if you do then you need to use the following syntax:

file:///G:/Work/gate-binary/gate-5.1-stable-3431/plugins/Ontology_Tools

1.3 Rename components:
The version of GATE on the destination machine might be different. Many plug-in names were changed from GATE 5.0 to 5.1, so you might find yourself porting an application from an older version of GATE to a newer. For example the plug-in "NP_Chunking" changed to "Tagger_NP_Chunking".

So you need to change this line from:
$relpath$../../../C:/Program%20Files/GATE-5.0/plugins/NP_Chunking
to:
$relpath$../../../C:/Program%20Files/GATE-5.0/plugins/Tagger_NP_Chunking

1.4 Remove components:
Sometimes you just need to remove components that you do not need. The following block describes a datastore:

which can be safely removed all together.

2. Other resources:
Other resources include for example your Jape files. If you are using a standard GATE gazetteer then your gapp file refers to a def file which you also need to relocate. If you are using an OntoGazetteer then again your gapp file will refer to an ontology file and a map file that you need to package too.

I have found a new library that deals with RDF for .NET: www.dotnetrdf.org

Интересувам се от "бионика", да го наречем.

Филмът "Ghost in the Shell" и факта, че имаме ЕЕГ апарат(баща ми е невролог) ми дава основата да мечтая за тези технологии. Нека под "бионика" разбираме способността на човешката нервна система да комуникира с външни прикирепени устройства - протези. Става въпрос за контролиране на изкуствени крайници като ръце и крака, а също устроиства които възтановяват зрението. ЕЕГ апарата е (скъпо) устройство, което разчита електромагнитните импулси от мозъка. Тези импулси са много слаби и точно ЕЕГ апарата действа като усилвател. Електроди се поставят най-близо до черепа (без да се отваря - просто ти ровят в косата докато стигнат скалпа) и след това се използва и паста, която дава по-добра проводимост. Може да се прави ЕЕГ и като се отвори черепа - тогава сигнала ще е най-добър естествено.

Нека да обобщя - сигналите от мозъка могат да бъдат разчитани, може да е трудно, но компютрите допринасят за това. Възможно е дори компютъра да се адаптира към нервните импулси на даден човек. В информатиката автоматичната адапатация се прави чрез алгорими за невронни мрежи . Разбира се тези невронни мрежи са под формата на програма, която имитира функцията на човешкия неврон. Тук нещата не трябва да се смесват - тук не говоря за физически неврони, които могат примерно да се имплантират в човека, а за понятието неврон в информатиката. Компютърният неврон съществува само в рамките на софтуер.

Един вариант, който си мисля, е за създване на изкуствена ръка чрез хидравлика. Като интересното е, че за течност на хидравликата може да се използва кръвта. Хидравликата може да доведе до това, че да е нужен само един мотор, който да променя налягането и по този начин да се постигат различи състояния на ръката. Може да звучи малко кофти да се ползва кръв, но нека видим предимствата. Циркулиращата течност освен за хидравлика може да се използва и като източник на енергия - иначе ще трябва да зареждате винаги ръката си с ток, а батериите не са много леки (все още) и не държат дълго (все още). От една страна енергия може да се произвежда от самото движение на кръвта, но няма как да е много или ще затрудним сърцето. Другият вариант е да се използва топлината на кръвта. Има примерно стирлингови двигатели, които работят с човешка топлина и въртят перка само от топлината на дланта. Стирлинговите двигатели са типичният за света на механиката начин за преобразуване на топлина в енергия, когато има значение КПД-то. Примерно при колите няма толкова значение КПД-то и затова те не се ползват там. Проблемът обаче в случая е, че големината на двигателя е доста на фона на произведения ток.
Нанотехологиите ще променят всичко - ще бъде създаден такъв електропроводников елемент, който да конвертира топлина в електричестово - вероятно на първо време с ниско КПД. После кръвта ще преминава през нещо с много голяма повърхност (заемащо малко обем) покрито с този електропроводников елемент и тази комбинация от голяма площ и малки размери ще дава нужната енергия в крайна сметка. Може електричестовото да се увеличи като се пусне някаква мрежа в тялото на човека, която да играе същата роля на "орган", който преобразува топлината на тялото в ток, ако кръвта преминаваща през ръката не дава нужното електрозахранване. А може и ръката да е покрита с боя, която допълнително оползотворява падащата светлина (има такова нещо - click here).

Темата просто е интересна.

По-долу може да видите low temperature Stirling engine:

Преди да запчнете с приготовленята за зелето, трябва да сте изпили бутилка Becks, 30 грама ракия, да сте замезили с 100 грама трушия и . . . 100 грама варени кестени. Това е тествания вариант - зелето става.

Абе, можше и без тая част - but it feels good, doesn't it?

Зелките се слагат цели та да има за сърми. Кочана се изрязва - прави се конусовиден разрез. Набучват се няколко пъти (с нож) да влезе солената вода, но не прекалено. Сложихме и няколко половинки.

Слага се още:
- 1 царевица (преварява се за по-сигурно)
- 2 дюли (нарязани)
- 1 цвекло за цвят(нарязано на няколко парчета)
- 2 ябълки (нарязани)
- 2 корена хрян(настърган)
- 1 кг морска сол (разтворихме я преди това колкото можеше)
- 1 речен камък (наследствен от Търново)

Нашият бидон е към 70 литра. Сложихме му чучурче за преточване, но се наложи Митко да го смени. После Митко преточи зелето доста пъти.
Държахме го два месеца на балкона и го почнахме. Сложихме му натриев бензоат да спре процеса на ферментация по 1мг на 1л. Камъка може би също има значение за втасването на зелето (мих го преди да го сложим).

Много добро стана. Хапваме го с олио, червен пипер и ракия.

Извадихме листа(добре оформени) и ги напълнихме с кайма, суджук и ориз. Овкусихме сместа преди това с ароматни подправки и вегета. Поставихме сърмите свити и притиснати една до друга в тефлонова тенджера. Покрихме сърмите с вода. Всико това къкри 20 минути и накрая добавихме половин буркан с домашни домати (от село Емен). Варихме още 10 минути и започна плющенето. Друго е когато сам си направиш зелето. Снимки нямаме ... трябваше да ядем. Сърмите бяха разкошни и преядохме ...

Life is expensive, but at least you get a free trip round the sun.

Das Leben ist teuer, aber zumindest erhalten Sie eine kostenlose Reise um die Sonne.

This post explains some steps of how to download and set up the GATE source code. This is needed when you would like to improve something.

SVN checkout: https://gate.svn.sourceforge.net/svnroot/gate

Open Eclipse in a new workspace (recommended). Use File->Import->Existing Projects into workspace->select your gate source dir->Finish.
Use the "Java Element Filters" to hide all "Non-Java elements" to make your project more compact.

Update:
My problems were related to an error that prevented me from downloading all GATE files while svn checking out the source. The problem is due to the fact that a filename allowed on Linux is not allowed on Windows. It was about ".cow:no-iframe" and ":" is not allowed on Windows. This halts the entire svn checkout and made me do all sort of tweaks and patches. The GATE source is over 500 MB and 13 000 files, so make sure you have everything before trying to fix it like me. If you are having problems like me, you should try a checkout on Linux to see if it is an OS dependency. If you copy source from Linux to Windows then you need to check in Eclipse that Properties->Resource->Text File Encoding->other is set to "UTF8".

I am interested in refactoring the code of the Othomatcher. This is the processing resource that matches all annotations of the same entity that they are indeed referring to the same thing. This is needed when a person or an organization is mentioned in different forms in the same document.

След като публикуваха линк на сайта на GATE към блога ми:

[15:02:53] Крум: ми, Тончо, да вземеш да почерпиш ли...
[15:03:14] Филип: eeee
[15:03:25] Филип: да не се надуеш сега
[15:03:26] Филип:
[15:03:36] Филип: браво
[15:08:34] аз: ами ... направо се пръскам честно казано
[15:08:50] аз: тука овиках света, питай другите
[15:08:52] аз:
[15:09:15] Ясен: verno verno, dopredi malko be6e na prozoreca i vika6e

This song and image are from Ghost in the Shell - "Lithium Flower". I love this song as I find it both romantic and energetic.


This song and image are from Ghost in the Shell - "Origa - Rise".


This clip is from the new game "Borderlands", song is "Cage The Elephant - Ain't No Rest For The Wicked". You can watch the original clip here with better sound.

Summary:
This article covers some of the basics of Sesame and how to deploy Ontotext OWLIM as a semantic repository.
You also need a copy of OWLIM(http://www.ontotext.com/owlim) semantic database from Ontotext. OWLIM comes in two favors: Swift OWLIM and Big OWLIM. Here the Big-OWLIM version 3.2.5 has been used. Big-OWLIM is the more robust and scalable version of OWLIM.
Currently this is not a guide to optimize your OWLIM performance, it just tells you how to install/use OWLIM. In order to configure OWLIM for best performance you need to read the documentation and/or consult Ontotext.

Without installing Sesame
This means to use only the content that comes with the OWLIM archive.

Actually when I first downloaded OWLIM I started searching for something like "start-server", but (from the OWLIM archive) there is no such thing. You can only use Sesame and OWLIM as libraries, not as a server waiting on a specific port. This is because OWLIM comes with some of the Sesame libaries, but not all. This on the other hand keeps the the OWLIM package size small - only 4MB. Also you may want a different version of Sesame.

You can use the "example.cmd" in the "getting-started" folder. This way by using various configuration files you can load ontologies, set queries to execute and do this without writing a single line of Java code. This use of OWLIM as a library is helpful when you would like to complete some task and release the memory.

Installing OWLIM in Sesame
The benefit of this option is that by having Sesame running as a "server" you actually use Big OWLIM as a "server" in the sense that we are all used to when using a relational regular database. Sesame allows support for external storage and inference engines. This allows you to use the same API (the Sesame API) and use different back-ends(like Big OWLIM for example).

First step is to download Aduna OpenRDF Sesame 2.2.4. There are also newer versions of Sesame, but in this article version 2.2.4 is recommended.
Let's refer to your Sesame installation folder as "SESAME_INSTALL"

Note that you may need to add the Lucene indexing jar file to the SESAME_INSTALL\lib folder. For Big OWLIM 3.2.6 for example version Lucene version 2.9.1 is required to be in your classpath.

What we do next is to enable enable Sesame to work through Tomcat as a server.
Next we deploy the Workbench which is the web UI that we would like to use.

1. Copy the following wars:
openrdf-sesame.war
openrdf-workbench.war
from
(Windows): %SESAME_INSTALL%\war
(Linux): $SESAME_INSTALL/war
to your Tomcat webapps folder.

1.1 Start the Tomcat server
(Linux):NOTE: the user which starts the Tomcat server, must be the same as the user which you logged-in to perform the deployment of Big OWLIM.

2. Sesame uses an application folder to store its files:
(Windows): %APPDATA%\Aduna\
(Linux): $HOME/.aduna

From the command prompt execute:
(Windows): echo %APPDATA%\Aduna\
(Linux): echo $HOME/.aduna
to see the full path.
If you have never used Sesame (just downloaded) maybe these directories are still not created!

To configure the Sesame Workbench go to:
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame"
(Linux): $HOME/.aduna/openrdf-sesame
and then check for "templates" folder. You need to create this directory if it does not exist. You must put in it a repository descriptor file for your new repository. You have to use the "bigowlim.ttl" from Big OWLIM installation folder as a template.

3. Then you need to add the owlim-big-3.2.5.jar from Big OWLIM installation folder to:
(Windows): "%TOMCAT_HOME%\webapps\openrdf-sesame\WEB-INF\lib"
(Linux): $TOMCAT_HOME/webapps/openrdf-sesame/WEB-INF/lib

4. Open http://localhost:8080/openrdf-workbench to get to know the Sesame UI workbench.

And here comes the bad surprise - OWLIM won't appear in the “new repository” form in the “Sesame Workbench” UI as available Semantic Repository type!

How to enable OWLIM in Aduna workbench
Summary:
Here will learn more details about how Sesame works and the basic configuration files you would need.
The key moment here is that we are going to use Sesame from another client. This is the console client. We are going to create an OWLIM repository through the console client and then this repository will be usable through the Sesame Web Workbench.

First you need to do the above procedure, but this time you need to copy the bigowlim.ttl in:
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\templates"
(Linux): $HOME/.aduna/openrdf-sesame-console/templates

and you need to put owlim-big-3.2.5.jar file in SESAME_INSTALL\ib folder.
We are doing this to make the console client aware of OWLIM.

Now there are two ways to continue to create an OWLIM repository and make it available in the Sesame UI workbench:

1.1 Start the OpenRDF Sesame console with:
(Windows): %SESAME_INSTALL%\bin\console.bat
(Linux): $SESAME_INSTALL/bin/console.sh
This way you connect to the default repository in
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\"
(Linux): $HOME/.aduna/openrdf-sesame-console

1.2 Connect to the openrdf-sesame.war that you have loaded in Tomcat by entering this command in the console:
connect http://localhost:8080/openrdf-sesame.

1.3 Now you can create a repository which will use
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame\repositories\"
(Linux): $HOME/.aduna/openrdf-sesame

as this is the repository configured for the openrdf-sesame.war with the following command:
create bigowlim.
Do not add the "ttl" extension.

Alternative path to enable Big OWLIM in Sesame workbench.

2.1 Start the OpenRDF Sesame console with:
(Windows): %SESAME_INSTALL%\bin\console.bat
(Linux): $SESAME_INSTALL/bin/console.sh

This way you connect to the default repository in:
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\"
(Linux): $HOME/.aduna/openrdf-sesame-console

2.2 Issue the command:
create bigowlim.

to create the new repository. Do not add the "ttl" extension. So now your new repository is in the console config/storage folder
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\repositories\"
(Linux): $HOME/.aduna/openrdf-sesame-console/repositories

,but you need it in the config/storage file

(Windows): "%APPDATA%\Aduna\OpenRDF Sesame\repositories\"
(Linux): $HOME/.aduna/openrdf-sesame/repositories

because this is the one used by the Sesame Workbench.

2.4 Copy your newly created repository from:
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\repositories\"
(Linux): $HOME/.aduna/openrdf-sesame-console/repositories

to

(Windows): "%APPDATA%\Aduna\OpenRDF Sesame\repositories\"
(Linux): $HOME/.aduna/openrdf-sesame/repositories

Next start again the workbench http://localhost:8080/openrdf-workbench and your new OWLIM repository should be ready to use (you have already created it in the previous steps).

So the bottom line is that you use the Sesame console with config/storage
(Windows): "%APPDATA%\Aduna\OpenRDF Sesame console\"
(Linux): $HOME/.aduna/openrdf-sesame-console

to create an OWLIM repository and you use

(Windows): "%APPDATA%\Aduna\OpenRDF Sesame\"
(Linux): $HOME/.aduna/openrdf-sesame

config/storage with the Sesame workbench.

Special thanks goes to:
Zdravko Tashev
Danail Kozhuharov