Archives de catégorie : dev

Openstreetmap: Generate heatmaps using Osmosis

I thought it would be nice to create heatmaps from the Openstreetmap data (i.e. for displaying the areas lacking « addr:housenumber »). I created, OsmosisHeatmap (jar file, source), a plug-in for Osmosis (a command line tool for processing Openstreetmap data) that generates a JavaScript file with the coordinates, JS that can be used by the sample HTML page based on Leaflet and Leallet.Heat.

And it gives something like that:

sample

Problème d’affichage du texte dans Visual Studio Code sur Kubuntu 16.04? Changez de police!

Après la mise à jour de Kubuntu 15.10 à la version 16.04, le texte avait une sale tronche: il était plus sombre et avait l’air moins précis.

En fait cela venait du fait que la police par défaut était devenue « Courier New ». La solution, qui n’avait rien à voir avec la couleur du texte (!), a été de modifier le fichier de réglages (menu Fichier / Préférences / Paramètres utilisateur) en rajoutant un entrée comme:

"editor.fontFamily": "Liberation Mono"
"editor.fontFamily": "Oxygen Mono"

J’ai galéré pour trouver pourquoi cela ne touchait que Kubuntu 16.04 (que ce soit sous KDE ou XFCE), et pas Ubuntu 16.04 (sous Unity) ou Ubuntu 15.10 (sous KDE): grâce à des VM j’avais comparé (avec « strace ») les différents fichiers chargés par vscode, sans réussite: il semblait que les fichiers étaient les même, notamment les fichiers de thème de GTK 2.

En fait, j’aurais pu trouver plus rapidement en utilisant des outils de dev de vscode (ceux de Chromium; on les active dans le menu aide et on les ouvre avec F12): j’aurais pu voir quel était la police utilisée en pratique dans l’éditeur.

Text display issue in Visual Studio Code in Kubuntu 16.04? Change the font!

After upgrading Kubuntu from 15.10 to 16.04, the text really looked ugly: it was darker (even when my theme specified « #ffffff ») and looked imprecise.

In fact, the reason was the default font which was now « Courier New ». The solution, which had nothing to do with the text color (!), was to modify the configuration file (menu File / Preferences / User Settings) by adding an entry like:

"editor.fontFamily": "Liberation Mono"
"editor.fontFamily": "Oxygen Mono"

I struggled to find why it only impacted Kubuntu 16.04 (either KDE and XFCE), and not Ubuntu 16.04 (on Unity) or Ubuntu 15.10 (on KDE): using some VMs I compared (with « strace ») the files loaded by vscode, without any success: it seemed that the files were the same, including the GTK 2 themes.

In fact, I could have found more quickly by using the vscode (Chromium) dev tools (unlock them in the Help menu and press F12): I would have seen which was the font actually used by the editor.

« Lazarus: Form Recovery » makes forms submit slow :-(

« Lazarus: Form Recovery » is a Firefox add-on that saves the text of the forms so it can be restored later (very handy, for example if you have typed a long mail/message/Wikipedia contribution and something bad happens, like a crash or a computer shutdown).

Unfortunately, I realized that it causes a small delay (something like 1.5s) when submitting a form. It is perceptible in the case of an Ajax form (or even an « offline » form where nothing is sent but data is processed locally in Javascript; in this case the result should be instant). I spend some times searching around, it was crazy! No problem with a simple « onClick » but only with « onSubmit ». Was it the debug version of React? No! Or the fact that I used the Javascript compilation of React JSX? No! There was nothing special in the Performance analyser. Then I realized that Chromium was fast. And Firefox dev edition was fast. Then I began the Firefox add-on hunt…

That’s strange that saving is not asynchronous…

(by the way, if you want to compile JSX files using Node.js, don’t install « node » on Ubuntu, that’s not « Node.js »! Install « nodejs » instead, and make a symlink from « /usr/bin/nodejs » to « node » somewhere on your path. Otherwise you may ask yourself why the JSX compiler does nothing. It’s just that the executable « node » is completely unrelated)

Atom: police japonaise / Japanese font

(English below)

Dans l’éditeur de texte Atom, sous Kubuntu 15.04, les caractères japonais s’affichaient comme un carré (□), alors que les autres éditeurs (Emacs, Kate, Geany…) n’avaient pas ce problème. Pour obtenir un affichage correct, j’ai mis comme police dans les préférences «Oxygen Mono, TakaoPGothic». Par contre, comme avec les autres éditeurs de textes, les caractères japonais ne s’affichent pas avec une longueur fixe (vu que la police TakaoPGothic n’est pas une police à chasse fixe), mais ce n’est pas grave.

In the text editor Atom, in Kubuntu 15.04, the Japanese characters were displayed as a square (□), whereas the other editors (Emacs, Kate, Geany…) behave correctly. In order to obtain a correct display, I set in the settings the font as «Oxygen Mono, TakaoPGothic». However, as in the other editors, the japanese characters were not displayed with a fixed length (given that the font TakaoPGothic is proportional), but that’s not really a problem.

 

Postgresql, micro-benchmark: inserting or not a blob

Let say that I have a table with 20 non-null text columns (32 characters long), 20 non-null integers and a blob (bytea, nullable). When inserting records, what is the overhead of setting the blob? Answer: about no difference up to 80ko. From that it get slower.

Testing details: 2000 records (on 2000 transactions), inserted on 16 threads, each of them having a persistent connection. The DB server is on the same machine (Ubuntu amd64), no particular configuration.

MyKanji 0.3

J’ai écrit une petite extension pour Firefox permettant de voir dans une page les kanjis que je n’ai pas encore appris. On peut l’installer depuis cette page (pas encore vérifié par Mozilla), et les sources se trouvent sur GitHub.

Comment cela fonctionne-t-il?

  • Phase 1: installation et configuration
    • installez le module
    • mettez dans le presse-papier un texte contenant les kanjis que vous connaissez. Par exemple en faisant un export texte d’un paquet Anki, et en copiant le contenu de ce fichier texte.
    • cliquez sur le bouton «文» dans la barre d’outils.
    • dans la zone de texte, coller le texte que vous avez préparé
  • Phase 2: utilisation
    • dans une page web, sélectionnez du texte ou toute la page (control-a)
    • ouvrez le menu contextuel (clic droit) et sélectionnez «Mes Kanjis»
    • vous obtiendrez la liste des kanjis inconnus, avec leur niveau

Microbenchmark: Expat outperforms Java and libxml2 in SAX parsing

I needed an XML parser for my tests with Rust (parsing a gigantic 2Go Wikivoyage XML dump) and there is no native one. So I would need a wrapper around a C implementation. At first, I used libxml2, because it has a very appealing name it has a very convenient XmlTextReader API where you are controlling the parsing loop (no callback like in SAX).

I wondered how it compared to the StAX API of Java, and made a simplistic test printing (to /dev/null) the names of all the nodes. And surprise, StAX was way faster:

  • the StAX (openjdk-7, Ubuntu 14.04, amd64)
    • 35 seconds
    • Memory usage: 30 Mo
  • XmlTextReader API of libxml2 (2.9.1), g++ 4.8.2
    • 50 seconds
    • 3.5 Mo

Then I made tests with SAX, comparing the default Java implementation with libxml2 and finally also libexpat 2.1:

  • Java: 30s (39Mo)
  • libxml2: 34s (2Mo)
  • libexpat: 21s (180ko)

Conclusions:

  • the Java implementation seems quite good (at least in this particular scenario)
  • libexpat is quite fast, but it’s SAX and not pull-parsing (which is very convenient)
  • if you are processing huge XML files, forget about libxml2

i18n: text expansion for several languages

From the KDE translations, I compiled statistics for text expansion, following the model presented by IBM on this page. So it will tell you for example that if you want to translate a text from English to French, which is between 11 and 20 characters long:

  • if you want 50% of such texts to fit, you will need 24% of additional space (for an English text of 100 pixels, you will need 124 pixels)
  • if you want 80%, you will need 57% of space
  • for 90%: 77%
  • for 95%: 98%
  • for 99%: 150%

Sources and data are available. Please excuse me for the poor HTML design (written in HTML 5, looks like HTML 1.0 ☺), I did not put any effort in it (not my cup of tea), but at least it does the job.

I also wrote another version of the generator using the Go language. I was a little bit disappointed because:

  • I did not get a formidable boost in performance (I should have, I have a 4-core CPU and Python can not fully take advantage of it because of its GIL). In fact, the file parsing part (the most time consuming one) gave similar performance, and the statistic production part was notably faster
  • The GTK/Pango (I used to get text length in pixels) binding for Go was incomplete and boring to complete (even if Go has a really nice support to write C bindings). I also had some crashes (related to GObject reference counting).
  • There is a port of Freetype written in Go, but the code to compute the length of a text was incomplete (it did not cover a given case)
  • At the time I wrote this version, I thought about integrating Fontconfig directly in the program, but it would have taken more time that I wanted to spend on it.