Author Archives: Avinash Meetoo

About Avinash Meetoo

Founder of Knowledge Seven. Linux, Android and Apple geek. Blogger, amateur musician & photographer.

Splitting a large text file on every blank line

I had a file of about a thousand lines containing the full description of multiple products (audio plugins from Plugin Alliance in my case). The file starts like this:

249
19.99
Mixland
Vac Attack★★★★★
A warm harmonically-rich optical limiter for compression with tube saturation that’s great on vocals, drums, basses, and your stereo bus.
Hardware EmulationsHidden GemsPA MembersSale

89.99
29.99
Woodlands Studio
VOXILLION★★★★★
A stunning and sophisticated, and complete vocal chain in one streamlined workflow. Featuring a high-end blend of a tube-driven preamp, two types of compressors, Nasal Dynamic EQs, Harmonics and more
Hidden GemsLimited Time OnlyPA_EXTSale

279
39.99
ADPTR AUDIO
Metric AB★★★★★
The mastering engineer’s best friend: Compare your mix to your favorite reference tracks. See,hear and learn even more with new expanded features.
Hidden GemsMasteringSale

and ends like this:

FreeDownload
PA FREE
bx_shredspread★★★★★
Intelligent M/S width for doubled riff guitars. Auto-avoid common phase problems, and sound extra-wide and tight!
FREEGuitar & BassHidden GemsM/S InsideMade by BX

FreeDownload
PA FREE
bx_tuner★★★★☆
Accurate tuning for guitar & bass with useful features like volume dimming. Tune up right before you hit the record button.
FREEGuitar & BassMade by BXSale

FreeDownload
PA FREE
bx_yellowdrive★★★★★
Warm to crunch to shred with this classic “Yellow” pedal in plugin form.
Creative FXFREEGuitar & BassHardware EmulationsMade by BXSale

What I wanted to do was to split the long file of a thousand lines into smaller files, each containing the lines corresponding to one plugin only.

Fortunately, the plugins were separated by an empty line. After some searching, I stumbled upon a Stackoverflow post of someone having the same issue and I settled on this solution enhanced by this comment:

gawk -v RS= '{ print > ("plugin-" NR ".txt")}' plugins.txt

As explained in the solution, “setting RS to null tells (g)awk to use one or more blank lines as the record separator. Then one can simply use NR to set the name of the file corresponding to each new record”. As explained in the comment, this will fail with the basic version of awk as it cannot handle too many open files at the same time (only 252 files were created with awk). Switching to gawk works great and now I have 277 files which I can further process.

Getting unknown terminal type warnings when using SSH?

I regularly SSH to my computers and servers and, on some of them, I was getting the following warning message:

'ansi+tabs': unknown terminal type.

This was just a warning and everything worked perfectly. But, still, having a warning message was not very, ahem, professional…

I have just found the cure: you need to have additional terminal definitions installed on the remote computer. On Red Hat / CentOS / Fedora, do:

dnf install ncurses-term

and on Debian, CentOS and derivatives:

apt-get install ncurses-term

and everything will be fine.

Convert multiple lines into one tab-separated line using paste

I generally have a lot of text files with such content:

1
Play track
Turn Around
Unlove this track
Panic Attack
Avinash Meetoo
110 scrobbles
2
Play track
Playing the Angel
Unlove this track
Precious
Depeche Mode
101
3
Play track
The Singles 86>98
Unlove this track
Personal Jesus
Depeche Mode
98

These are the three tracks I have listened to most since I started using Last.fm way back in 2005.

What I want to achieve is to easily convert these 21 lines (3 x 7) into 3 tab-separated lines which can then be easily imported into, say, Libreoffice Calc:

1<TAB>Play track<TAB>Turn Around<TAB>Unlove this track<TAB>Panic Attack<TAB>Avinash Meetoo<TAB>110 scrobbles
2<TAB>Play track<TAB>Playing the Angel<TAB>Unlove this track<TAB>Precious<TAB>Depeche Mode<TAB>101
3<TAB>Play track<TAB>The Singles 86>98<TAB>Unlove this track<TAB>Personal Jesus<TAB>Depeche Mode<TAB>98

For many years, each time I had to do that, I wrote a small Awk script. But, today, thanks to RudiC on the Unix Stack Exchange, I have the perfect recipe using paste:

paste -s -d "\t\t\t\t\t\t\n" file-containing-all-the-lines.txt

Naturally, you can adjust the number of “\t” if you have fewer or more fields.

Enjoy :-)

Running Puddletag on Linux distributions with limited Python 2 support

Leave a reply

I am a big fan of Puddletag, an audio tag editor for Linux similar to the Mp3tag Windows program. As seen above, Puddletag uses a spreadsheet-like layout which makes it quite unique in the Linux world and matches 100% with how I like tagging to be. Puddletag and I are on the same wavelength and I have been using it for some years now to curate my extensive music collection.

Unfortunately, Puddletag was written in Python 2 and relies on a number of Python 2 libraries. When I upgraded to Fedora Linux 32 a few days ago, I lost a number of those libraries as well as Puddletag (as Python 2 reached its end of life (EOL) on 1st January 2020).

Fortunately, the Linux community came to the rescue and created a corresponding AppImage which works on all new Linux distributions like Fedora 32 or Ubuntu 20.04. Get the AppImage here courtesy of Patsim and have fun!

I’ve tested it in Fedora Linux 32 and it works great!

Libreoffice with Flatpak: Adding dictionaries for other languages

Leave a reply

I generally write in either English, French or Mauritian Kreol.

I also use Libreoffice when I need a word processor or a spreadsheet (or, even, sometimes, a drawing software). Lately, I have discovered Flatpak (“The Future of Apps on Linux”) and Flathub (“An App Store for Linux”) and I am sold. Installing the latest version of Libreoffice from Flathub using Flatpak is a simple:

$ flatpak install flathub org.libreoffice.LibreOffice

I have noticed though that this command tends to only install the English dictionaries for spellchecking. I didn’t know how to install more dictionaries to Libreoffice so I asked on the official Flathub forum. And, fortunately, someone from the community pointed me towards the solution. The idea is to get some information about the Libreoffice installation:

$ flatpak info org.libreoffice.LibreOffice

LibreOffice - The LibreOffice productivity suite

ID: org.libreoffice.LibreOffice
Ref: app/org.libreoffice.LibreOffice/x86_64/stable
Arch: x86_64
Branch: stable
Version: 6.4.3.2
License: MPL-2.0
Origin: flathub
Collection: org.flathub.Stable
Installation: system
Installed: 686.0MB
Runtime: org.freedesktop.Platform/x86_64/19.08
Sdk: org.freedesktop.Sdk/x86_64/19.08

Commit: ddcb114395acb30f633a06fd065598ace0fbe4330c49a784a50911b0222f5f95
Parent: fa891c405f685f7859e2bb623b29db5cdb3e9e1d80d8c31f30a5d21edcc9a3eb
Subject: Update to libreoffice-6.4.3.2 (5a34256e) Date: 2020-04-16 18:32:28 +0000

Notice that the runtime is org.freedesktop.Platform/x86_64/19.08 which implies that org.freedesktop.Platform.Locale/x86_64/19.08 contains all the dictionaries. But, as pointed out by stbergmann in the forum,

The *.Locale extensions are special, in that flatpak by default only downloads that part of such an extension that matches the users current system locale, while the –reinstall hack unconditionally downloads all of it. There obviously needs to be usability improvements here, as this appears to be a common issue for users.

And this is the –reinstall hack he is talking about:

$ flatpak install --reinstall flathub org.freedesktop.Platform.Locale/x86_64/19.08

and, after a few seconds, the dictionary issue is cured. Now, in addition to English, you have French plus all the other languages. Enjoy :-)

Disable tracker in Gnome if you do not need it

Leave a reply

According to its website, “Tracker is a filesystem indexer, metadata storage system and search tool.”

“By using Tracker, you no longer have to remember where you’ve left your files. To locate a file you only need to remember something about it, such as a word in the document or the artist of the song. This is because as well as searching for files in the traditional way, by name and location, Tracker searches files’ contents and metadata.”

That’s great I guess for some people.

For me, Tracker is a massive pain as I have a lot of files, some of them huge and a lot of them being binary files (think images and photos, music files, videos, LibreOffice files, PDF documents, etc.) On my Fedora Linux installation, which is now running in a virtual machine on a brand new MacBook Pro, the Tracker extractor and indexer uses a lot of CPU resources. Too much in fact for something that I never ever use.

You see, I am an “informatician” (i.e. a computer scientist — a very bad designation — think calling a surgeon a scalpel scientist) and, as such, I make it a must to properly save all my files in folders and subfolders. I also name the files very consistently with dates, sensible use of words, etc. for proper sorting and, hence, searching.

So I don’t need Tracker as I pretty much can narrow down to the single file I need in a few seconds.

How to disable Tracker in Gnome

I found the following set of commands online which disable Tracker:

    gsettings set org.freedesktop.Tracker.Miner.Files enable-monitors false
    gsettings set org.freedesktop.Tracker.Miner.Files ignored-files "['*']"
    gsettings set org.freedesktop.Tracker.Miner.Files crawling-interval -2

Looking at the source code of Tracker,  the -2 indicates that Tracker should be disabled. The next step is to kill all current Tracker processes:

    pkill tracker

and, finally, clear the Tracker cache

    rm -rf ~/.cache/tracker

Playing Quake again after 23 years

Leave a reply

Quake was born when Christina and I were studying in France in 1996.

Like most people we knew, we fell in love with the game: the graphics were amazing (and we even bought a 3dfx Voodoo 2 to run it), the audio was great and the atmosphere phenomenal. As a matter of fact, Quake is one of the few games I completed. I ran it on Windows 95.

Fast forward 23 years

Today, my laptop runs Fedora Linux 29 and I only use its integrated Intel graphics chipset (despite the laptop having an nVidia chipset as well). This morning, I stumbled upon Quaddicted:

From there, I quickly discovered that, since Quake was released as open source software in 2012, a number of people have embarked on correcting bugs and enhancing the Quake engine. According to Quaddicted (and who am I to disagree!), the best engine for Linux is Quakespasm.

I quickly downloaded it and found an executable called quakespasm-sdl2 which I quickly ran. I got an error message telling me that a folder id1 was not found.

I remembered having zipped my Quake installation years and years ago and, because I am a very methodical guy, took only 5 seconds to find the ZIP. It had the id1 folder in it which I quickly copied to the Quakespasm folder. I got the Quake music. I ran quakespasm-sdl2 again and, lo and behold, Quake started in all its glory. I quickly selected the full screen option and opened a save game from decades ago and it worked! This is what you can see in the top screenshot.

But quake is more than quake…

While reading Quaddicted, a stumbled upon Quake Injector, a tool written in Java which allows you to easily download and run 3rd party maps in Quake. Think maps as levels, distinct from the Quake default levels, but as great and sometimes greater to play.

[Caveat: Quake Injector did not run at first as I had Java 11 on my Linux laptop. I had to switch to Java 8 (which I also had but was inactive)]

As soon as Quake Injector started, I installed the very highly rated DaMaul6 map and this is where I am now. Wish me luck as DaMaul6 is known to be very difficult.

A journey from Unison to Syncthing and back to Unison

Leave a reply

I generally work on my laptop, an old Dell Inspiron 15 still running Fedora Linux 29 quite well. At home, I also tend to work on an Apple iMac (still) running macOS El Capitan (which I intend to upgrade to High Sierra soon — unfortunately, Apple has decided that this iMac cannot run Mojave).

Since the very beginning, I needed to have a proper file synchronisation process for these two computers and cloud solutions such as Google Drive or Dropbox were not options as I had many many Gigabytes to synchronise and this would have been very expensive to be done online.

For 1-2 years, I used Unison, a file synchronisation utility written by one of my Computer Science heroes, Benjamin Pierce, author of the quite renowned book “Types and Programming Languages” which, I have to confess, I have not yet read because, well, I never had the chance to stumble upon it. Anyway, Unison is a tool which does two-way synchronisation between computers and every time there is an inconsistency (a new file, a deleted file, a change, etc.) ask the user to manually choose on the action to make. One important of Unison is that one has to run it manually i.e. the synchronisation only happens when initiated by the user. For example, I like to run Unison every afternoon when I get back home after one day of work.

One year ago, I stopped using Unison. The reason was that I wanted to have real time synchronisation between the two computers i.e. as soon as one changes, the other one is updated (if on of course.) I settled on Syncthing. I tried two different ways of using Syncthing (with a central repository or in peer-to-peer mode) and both worked as expected. But after one year, I have to say that I have two issues with Syncthing. First of all, changes are detected through the inotify mechanism provided by Linux and the macOS kernel. As soon as a file is changed, Syncthing knows about it and can propagate the change to the other computer. All good in principle. But this also means that all errors (a file deleted by mistake, a file overwritten by mistake, etc.) are also immediately propagated. So I used different kinds of versioning techniques to make sure that backups are kept in these situations. But, nevertheless, I slowly realised that real-time propagation might not be a good idea. The second issue is that Syncthing uses a lot of CPU.

Two days ago, I stopped the Syncthing service on all my computers and revived my Unison installation. And, believe it or not, Unison works great for my use case: I can synchronise my computers when I want (after work for instance) and I have the possibility (if there are incoherences) to think deep about which version to keep. For me, and because my files are very valuable to me, this is a much better process.

[At this point, let me point out that I use Unison for synchronisation and I use regular rsync for backup on other devices. It is important to understand the difference between these two processes. Synchronisation is not always needed but backups definitely. Especially when it is compliant with the 3-2-1 strategy.]

What about you? What do you use for synchronising multiple computers when cloud synchronisation is not possible? Do you use Unison? Syncthing? Something else? Why?

Informative and Restrained as opposed to Superficial and Flashy

Leave a reply

Infotech 2017 has started.

And I am happy to notice that, except for one or two stands, things are much more “Informative and Restrained” compared to previous editions where things tended to be “Superficial and Flashy”.

Allow me to explain.

In Mauritius, for the past few years, we have become a nation of seminars, workshops, conferences and exhibitions and, unfortunately, many of them are quite superficial and very very flashy indeed. For the past six months, I have been to many such events where the venue was beautiful (a nice hotel with a beautiful view of the lagoon), the food was excellent, the hostesses out of this world but where, personally, I felt that there was not much to listen to and learn from, except from a minority of the speakers. This is what I call “Superficial and Flashy”.

What I would prefer to have, from a personal point of view, is the kind of chaotic geekish meetup as pictured above. An event where intelligent people of all horizons can meet, exchange views, share ideas and move forward together. Of course, there is a need for a venue and some food but nothing ostentatious. This is what I call “Informative and Restrained”.

The thing is that it is easier to do “Superficial and Flashy” than “Informative and Restrained”. The reason for that is that to be informative, the speakers need to be of high-caliber and need to be properly prepared.

This is your typical Googler. Similar people are changing our worlds everyday at Google, Facebook, Amazon, Apple, etc. but also in the IT division of most of the companies in the world. And, before you laugh, let me remind you that they run the world.

Pictured above are some of the people who have basically built the world as it is known today. Without them, we would still be waiting for The A-Team to be shown on TV on Saturday night. They are Steve Jobs (Apple), Sergey Brin (Google), Bill Gates (Microsoft), Larry Page (Google), Mark Zuckerberg (Facebook) and Jeff Bezos (Amazon). The missing ones being Linus Torvalds (Linux) and Richard Stallman (Free Software Foundation).

Of course, we won’t have Steve (RIP), Sergey, Bill, Larry, Mark, Jeff, Linus or Richard at Infotech. Maybe next year…

But we’ll have the 2nd best thing: the (real) innovators of Mauritius, each on his/her respective “Informative and Restrained” stand and willing to share his/her passion with you.

You just have to put aside your tendency to value the “Superficial and Flashy”, walk toward them and talk to them.

Enjoy 🙂

(First photo, courtesy of Le Méridien. Second photo, courtesy of Concept7. Third photo, courtesy of Business Insider. Fourth photo, courtesy of Youth Connect. Fifth photo, courtesy of PC Risk).

How to reduce the amount of disk space used by the systemd journal

Leave a reply

We, Linux people, generally use systemd now and one of its components is the journal controlled by the journalctl command line tool.

As explained on the Arch wiki,

systemd has its own logging system called the journal. The /var/log/journal/directory is a part of the systemd package and the journal will write to /var/log/journal/

The journal is always appended and therefore grows in size. On my laptop, the journal was taking 1.8Gb of space and was full of details which, I believe, I’ll never need. So I decided to clear all old contents (which the systemd people call a vacuum). I issued:

journalctl --disk-usage
journalctl --vacuum-size=64M
journalctl --disk-usage

And the journal immediately became smaller. I then issued a

journalctl --verify

which made me realise that some of the remaining journal files were corrupted (for some reason). There is no journal repair tool in systemd so I simply removed the offending files (with rm).

Now, I can easily check my journal entries for today and I know everything will be all fine:

journalctl --since today