Tag Archives: education

Splitting a large text file on every blank line

I had a file of about a thousand lines containing the full description of multiple products (audio plugins from Plugin Alliance in my case). The file starts like this:

249
19.99
Mixland
Vac Attack★★★★★
A warm harmonically-rich optical limiter for compression with tube saturation that’s great on vocals, drums, basses, and your stereo bus.
Hardware EmulationsHidden GemsPA MembersSale

89.99
29.99
Woodlands Studio
VOXILLION★★★★★
A stunning and sophisticated, and complete vocal chain in one streamlined workflow. Featuring a high-end blend of a tube-driven preamp, two types of compressors, Nasal Dynamic EQs, Harmonics and more
Hidden GemsLimited Time OnlyPA_EXTSale

279
39.99
ADPTR AUDIO
Metric AB★★★★★
The mastering engineer’s best friend: Compare your mix to your favorite reference tracks. See,hear and learn even more with new expanded features.
Hidden GemsMasteringSale

and ends like this:

FreeDownload
PA FREE
bx_shredspread★★★★★
Intelligent M/S width for doubled riff guitars. Auto-avoid common phase problems, and sound extra-wide and tight!
FREEGuitar & BassHidden GemsM/S InsideMade by BX

FreeDownload
PA FREE
bx_tuner★★★★☆
Accurate tuning for guitar & bass with useful features like volume dimming. Tune up right before you hit the record button.
FREEGuitar & BassMade by BXSale

FreeDownload
PA FREE
bx_yellowdrive★★★★★
Warm to crunch to shred with this classic “Yellow” pedal in plugin form.
Creative FXFREEGuitar & BassHardware EmulationsMade by BXSale

What I wanted to do was to split the long file of a thousand lines into smaller files, each containing the lines corresponding to one plugin only.

Fortunately, the plugins were separated by an empty line. After some searching, I stumbled upon a Stackoverflow post of someone having the same issue and I settled on this solution enhanced by this comment:

gawk -v RS= '{ print > ("plugin-" NR ".txt")}' plugins.txt

As explained in the solution, “setting RS to null tells (g)awk to use one or more blank lines as the record separator. Then one can simply use NR to set the name of the file corresponding to each new record”. As explained in the comment, this will fail with the basic version of awk as it cannot handle too many open files at the same time (only 252 files were created with awk). Switching to gawk works great and now I have 277 files which I can further process.

Convert multiple lines into one tab-separated line using paste

I generally have a lot of text files with such content:

1
Play track
Turn Around
Unlove this track
Panic Attack
Avinash Meetoo
110 scrobbles
2
Play track
Playing the Angel
Unlove this track
Precious
Depeche Mode
101
3
Play track
The Singles 86>98
Unlove this track
Personal Jesus
Depeche Mode
98

These are the three tracks I have listened to most since I started using Last.fm way back in 2005.

What I want to achieve is to easily convert these 21 lines (3 x 7) into 3 tab-separated lines which can then be easily imported into, say, Libreoffice Calc:

1<TAB>Play track<TAB>Turn Around<TAB>Unlove this track<TAB>Panic Attack<TAB>Avinash Meetoo<TAB>110 scrobbles
2<TAB>Play track<TAB>Playing the Angel<TAB>Unlove this track<TAB>Precious<TAB>Depeche Mode<TAB>101
3<TAB>Play track<TAB>The Singles 86>98<TAB>Unlove this track<TAB>Personal Jesus<TAB>Depeche Mode<TAB>98

For many years, each time I had to do that, I wrote a small Awk script. But, today, thanks to RudiC on the Unix Stack Exchange, I have the perfect recipe using paste:

paste -s -d "\t\t\t\t\t\t\n" file-containing-all-the-lines.txt

Naturally, you can adjust the number of “\t” if you have fewer or more fields.

Enjoy :-)

Informative and Restrained as opposed to Superficial and Flashy

Infotech 2017 has started.

And I am happy to notice that, except for one or two stands, things are much more “Informative and Restrained” compared to previous editions where things tended to be “Superficial and Flashy”.

Allow me to explain.

In Mauritius, for the past few years, we have become a nation of seminars, workshops, conferences and exhibitions and, unfortunately, many of them are quite superficial and very very flashy indeed. For the past six months, I have been to many such events where the venue was beautiful (a nice hotel with a beautiful view of the lagoon), the food was excellent, the hostesses out of this world but where, personally, I felt that there was not much to listen to and learn from, except from a minority of the speakers. This is what I call “Superficial and Flashy”.

What I would prefer to have, from a personal point of view, is the kind of chaotic geekish meetup as pictured above. An event where intelligent people of all horizons can meet, exchange views, share ideas and move forward together. Of course, there is a need for a venue and some food but nothing ostentatious. This is what I call “Informative and Restrained”.

The thing is that it is easier to do “Superficial and Flashy” than “Informative and Restrained”. The reason for that is that to be informative, the speakers need to be of high-caliber and need to be properly prepared.

This is your typical Googler. Similar people are changing our worlds everyday at Google, Facebook, Amazon, Apple, etc. but also in the IT division of most of the companies in the world. And, before you laugh, let me remind you that they run the world.

Pictured above are some of the people who have basically built the world as it is known today. Without them, we would still be waiting for The A-Team to be shown on TV on Saturday night. They are Steve Jobs (Apple), Sergey Brin (Google), Bill Gates (Microsoft), Larry Page (Google), Mark Zuckerberg (Facebook) and Jeff Bezos (Amazon). The missing ones being Linus Torvalds (Linux) and Richard Stallman (Free Software Foundation).

Of course, we won’t have Steve (RIP), Sergey, Bill, Larry, Mark, Jeff, Linus or Richard at Infotech. Maybe next year…

But we’ll have the 2nd best thing: the (real) innovators of Mauritius, each on his/her respective “Informative and Restrained” stand and willing to share his/her passion with you.

You just have to put aside your tendency to value the “Superficial and Flashy”, walk toward them and talk to them.

Enjoy 🙂

(First photo, courtesy of Le Méridien. Second photo, courtesy of Concept7. Third photo, courtesy of Business Insider. Fourth photo, courtesy of Youth Connect. Fifth photo, courtesy of PC Risk).

How to reduce the amount of disk space used by the systemd journal

We, Linux people, generally use systemd now and one of its components is the journal controlled by the journalctl command line tool.

As explained on the Arch wiki,

systemd has its own logging system called the journal. The /var/log/journal/directory is a part of the systemd package and the journal will write to /var/log/journal/

The journal is always appended and therefore grows in size. On my laptop, the journal was taking 1.8Gb of space and was full of details which, I believe, I’ll never need. So I decided to clear all old contents (which the systemd people call a vacuum). I issued:

journalctl --disk-usage
journalctl --vacuum-size=64M
journalctl --disk-usage

And the journal immediately became smaller. I then issued a

journalctl --verify

which made me realise that some of the remaining journal files were corrupted (for some reason). There is no journal repair tool in systemd so I simply removed the offending files (with rm).

Now, I can easily check my journal entries for today and I know everything will be all fine:

journalctl --since today

World Creativity and Innovation Day

20160421-panelists

Every 21 April, we celebrate the World Creativity and Innovation Day in Mauritius.

This year, the Mauritius Research Council organised a workshop on Creativity and Innovation and the the ICT Advisory Council, which I preside, was given a slot of 30 minutes. I quickly prepared a few slides like I like them to be (beautiful, memorable and witty) and I asked Logan Velvindron and Ish Sookun, both members of the ICT Advisory Council, to join me in a panel on Creativity and Innovation among Youth.

20160421-albert-einstein

The first topic was about having fun when learning and creating. I mentioned that school and teachers can be boring, especially in our age of having Wikipedia and YouTube in our pockets. I said that it was time for our education system to become fun again, where young people can enjoy creating and learning a lot of new things in the process. Both Ish and Logan explained how fun they have everyday at work and how important this is for them.

20160421-steve-jobs

The second topic was about not to be afraid to be different. In Mauritius, we have a culture of conformism. Women have specific roles to play. Kids also. And, of course, everyone should remain at his place. This is bullshit! Our world is created by people who think differently and who are not afraid to take risks and disrupt existing establishments… Logan and Ish told the audience how they discovered computers, decided to become geeks and do things differently from others. They were very thankful to have supportive parents who didn’t try to impose anything on them.

20160421-linus-torvalds

The third topic was about being a geek and the value of open source software for a country like Mauritius. I explained that open source software is free and this is very important for Mauritius which is not a very rich country. But I also explained how having access to the source code of software is essential for Computer Science teachers like me to create the new generation of top programmers which Mauritius will need in the future. Young people cannot learn complex programming just by reading a book or listening to a teacher: they need to see real source code of real complex software. Logan and Ish explained how they got involved in open source software and how we all now form part of organisations such the Linux User Group of Mauritius and Hackers.mu and the value those organisations have.

20160421-sheryl-sandberg

The last topic was about being a doer rather than a talker and that aiming for perfection, while sometimes a good thing, can sometimes prevent us from achieving. This is something I have noticed over the years in Mauritius: we love our committees. Doers are not revered though. This mentality has to change. I told the audience that we are as good as anyone from anywhere, whether it is the USA or France or India. But we should stop focusing on doing speeches. We should identify small but important problems and find feasible solutions for them.

After the panel, I got a question from Anibal Martinez who is collaborating with the Government to set up an incubator. He asked me how we can solve the Computer Science education issue in Mauritius.

I told him that in the short term, a lot can be done with ad-hoc training courses, such as the ones that I provide at Knowledge Seven. But this is not a viable model for the country. In the medium to long term, state-funded universities should find a way to revamp their Computer Science departments, which are absolutely substandard at this moment, to produce an adequate number of very competent computer scientists every year. The industry is ready to move up a level but there are not enough competencies.

Thanks to Ish and Logan for participating in the panel.

10 reasons why Linux is better than Windows for the Mauritian School IT Project

20030718-top-10

1) Linux is free !

Linux distributions can be obtained freely and copied at will.

2) Linux is open source !

The sources of Linux and thousands of other open source applications are readily available. Learning to build software becomes easy just by looking at those examples.

3) Linux has a community of users !

Linux is not only an operating system but also an extensive community of users willing to share experiences and support each other. The Linux User Group of Mauritius (www.linux.mu), among others, represents this community in Mauritius. Extensive user documentation created by fellow Linux users exists on the Internet (www.tldp.org).

4) Linux runs thousands of software !

A Linux distribution typically contains loads of software (e.g. Redhat Linux 9 comes with 1402 software packages) and many more can be obtained on the Internet (e.g. on www.freshmeat.net and www.sourceforge.net). Most of these software packages (e.g. productivity, graphical, educational, software development or Internet applications) are free.

5) Linux is robust, secure and efficient !

Linux has been designed to be very robust (it recovers gracefully from the whole range of exceptional situations), secure (every user can have private files and specific privileges) and efficient (it can be used on previous generation computers like 486s and Pentium I). Some very well-known companies now use Linux for their processing needs (e.g. Amazon, Google, the NASA, the CIA and even our own Mauritian Servihoo).

6) Linux is based upon open standards and open protocols !

Linux implements most open standards and open protocols including but not limited to TCP/IP, HTTP, MathML, PNG, SOAP, Web Services, XHTML, XML, XSL…

7) Linux is compatible with everything else !

Linux can easily coexist and work with all kinds of alternative operating systems systems (like Unix, Windows, MacOS…). Linux applications can also read and write files in proprietary format (like .doc or PDF). Linux can even run some Windows applications (most notably office applications).

8) Linux is an excellent environment to learn programming !

A Linux distribution comes with numerous software development environment and compilers (for C, C++, Java, Python, TCL/Tk, Fortran, Ruby…).

Java 2 Enterprise Edition is also available through the freely available J2EE implementations from Sun or JBoss. With the availability of the two leading open source databases in most distributions, PostgreSQL and MySQL, a student has all the tools required to learn to build extensive enterprise software applications.

9) Linux can be easily administered remotely !

Every computer running Linux can be administered remotely using the very secure SSH protocol. This means that all eventual maintenance and installations can be done remotely. This implies that every school need not have a local system administrator.

10) Linux is being promoted by some very well-known companies !

IBM (www.ibm.com/linux), HP (www.hp.com/linux), Oracle (www.oracle.com/linux), SAP (www.sap.com/linux), Sun (www.sun.com/linux) and other well-known companies are heavily investing and developing products for Linux.

This clearly shows their trust in Linux.

Linux is the future !

Thanks to all those who have contributed ideas and to Yash Nursinghdass in particular.

Copyright Avinash Meetoo | Published under the Gnu Free Document License