The New Colossus

Not like the brazen giant of Greek fame,
With conquering limbs astride from land to land;
Here at our sea-washed, sunset gates shall stand
A mighty woman with a torch, whose flame
Is the imprisoned lightning, and her name
Mother of Exiles. From her beacon-hand
Glows world-wide welcome; her mild eyes command
The air-bridged harbor that twin cities frame.

"Keep, ancient lands, your storied pomp!" cries she
With silent lips. "Give me your tired, your poor,
Your huddled masses yearning to breathe free,
The wretched refuse of your teeming shore.
Send these, the homeless, tempest-tost to me,
I lift my lamp beside the golden door!"

Download the Dorico Trial

Dorico, the new music engraving software from Steinberg is now available for download as a trial. If you wish to avoid Steinberg’s intrusive download assistant, you can use the links below:

Dorico for Windows 1.0.10

Dorico for MacOS 1.0.10

Be aware though that this is a 8.4GB(!) download, so make sure your browser supports resumable downloads. Otherwise, if you are handy with a command line shell (and with bash available on Windows 10 this is also useful for Windows users), you can try downloading Dorico using wget or curl:

# with curl:
$ curl -C - -O

# with wget:
$ wget -c

Teaching a Computer to Engrave Music part I

This is the first of a series of articles in which I explore current and future solutions for digital music typesetting.

In the last few months, I’ve been toying around with the idea of writing my own software that can engrave/typeset music. It is not only that I find all of the current software solutions lacking in some way or another, but the process of typesetting music on a page or a device screen is so involved and goes to the heart of what music is, it’s almost irresistibly fascinating.

But first, let’s discuss what’s wrong with current music notation software, and start perhaps with the elephant in the OSS room, Lilypond (since that’s also what I have used to typeset music for the last ~10 years). Lilypond is without doubt the open-source software solution to music typesetting. It’s probably best-known for its high-quality output, as well as for the fact that it requires users to input musical content as text, in a non-interactive manner. Before I list my problems with Lilypond, I’ll start by praising it (after all, it’s the tool I use to make my bread).

Lilypond’s default quality of engraving is really really good, especially if you compare it to the two main commercial software packages, Finale and Sibelius. Lilypond is about 20-years old and represents countless hours of coding donated by a large group of programmers, all devoted to the idea of providing a free high-quality tool for typesetting music. But Lilypond has many shortcomings when used professionally:

1. Non-interactive workflow is a pain in the butt

There’s really no way about it. The WYSIWYM workflow imposed by Lilypond is more suited to a programmer mindset than that of a musician. It’s true that there are some tools designed to improve this state of affairs, such as Frescobaldi or my very own Lydown, but they do not and cannot really provide an adequate solution.

Note entry is easy enough, but the hard part is reviewing the music for mistakes, putting in good page breaks, and performing all sorts of tweaks to appearance, like nudging slurs etc. Each of these operations requires a recompilation of the score, which is a lengthy process of trial-and-error for any musical piece of substantial size. Twiddling your thumbs while waiting for Lilypond to update your score, in order to see a slur moved by two millimeters gets old really fast.

2. Anything out of the ordinary requires programming

Lilypond is really amazingly flexible, and at the hands of an accomplished programmer can be made to do absolutely anything, but most musicians, yours truly included, don’t really feel like writing computer code when putting together a score.

Just to give you some examples of what kind of problems require programming, here’s a selection of Lilypond mailing list discussions from recent weeks:

Hell, even stuff like slur tweaking requires a bit of code.

To make matters worse, programming in Lilypond is done using Scheme, a dialect of Lisp, which is arguably not the most accessible choice for non-professional programmers. It is also apparently no very much future-proof.

3. Source files as programs, not documents

The Lilypond syntax is quite good for expressing musical ideas, but no matter how you look at it, the Lilypond files prepared by users are actually computer programs that use a special DSL to express music, rather than proper documents containing musical content. This is also the reason why Lilypond files cannot really be automatically processed by third-party tools (see also below), as they can contain code that does anything from custom tweaking of musical symbols to dynamic generation of musical content.

4. No separation of content and presentation

Another problem which complicates the editing of music is the fact that Lilypond provides no easy way to separate the music itself and certain details of its graphical representation, like system and page breaks, various overrides or tweaks to slurs etc.

Since any override, such as explicit system breaks, needs to be placed inside musical expression, right between notes, it complicates the editing of source files and makes them less readable.

5. Text files are inadequate for editing polyphonic multi-part music

With Lilypond you can structure your input text files any which way you’d like, but no matter the structure chosen, you won’t be able to efficiently edit polyphonic music. Finding the source for a wrong note in the violino I part in bar 231 is not so easy. It’s true Lilypond provides point-and-click links in its PDF output, but at least on my Macbook this feature is broken.

In addition, because each part is (usually) represented as a contiguous stream of music, editing a single bar or moment in the score, and moving from one part to another requires moving to a completely different place in the source file, or even moving to an entirely different source file, depending on how the source is structured.

6. Separate entry for lyrics, figured bass, chords

Related to the previous point, lyrics, figured bass, chords, are all entered separately from the notes themselves. For figured bass, for example, you also need to enter durations, effectively doubling your work. Lyrics can be either entered with durations or automatically aligned to notes based on beaming, slurring and ties. Add or delete a tie, change beaming, and your text alignment is screwed.

7. Compilation is slow

Compiling a score for, say, a moderately-sized Bach cantata using Lilypond is a matter of 30 to 60 seconds. Add to that compilation of parts and your looking at a good 2-3 minutes for each time you want to create PDF’s of your music (personally I prefer not to keep PDF files - you can never be sure if they’re up to date).

8. Documentation and support is lacking

The Lilypond documentation goes to great detail in order to instruct users, but finding solutions to problems is often a case of hit-and-miss. Add to that the fact that many problems faced by users are solved by some programming, which leads users to seek solutions on the Lilypond mailing list, which is really a poor form of support.

This state of affairs is also caused by the fact that sharing code, and building tooling around Lilypond, is not very easy for novices. Professional users who are seemingly adept at linux would no doubt always find a way to create a bunch of tools related to their own Lilypond workflow. But the lack of any minimal framework for Lilypond, a sort of “standard library” of tools, means that users are left to their own devices. They can ask for advice, but there’s no accessible, automated system for sharing code. (Full disclosure: I’m the author of lyp, a package manager for Lilypond, but has yet to receive any embrace from the Lilypond community.)

9. Poor interoperability

Although Lilypond is open-source software, it doesn’t really play nice with other software. Tools that wrap Lilypond like the Frescobaldi IDE, or drive it like my own Lydown and Lyp have never really gained any substantial following in the Lilypond community, which seems totally fine with being an isolated little Stallman-esque island of musical geeks with a who-cares-if-they-use-it attitude.

I find this state of affairs off-putting, as well as counter-productive. I believe Lilypond could have been made into a much more comprehensive tool just by making it work better with the outside world. As stated above, Lilypond is incredibly flexible, but that is both a blessing and a curse. Code-reuse? You’re on your own. PDF Post-processing? You’re on your own. Efficient framework for structuring your files? You’re on your own. API for generating musical scores from third-party tools? You’re on your own.

The result is, that while the Lilypond community is doing fine, my impression is that Lilypond is slowly being more and more marginalized by both the programmer community (witness for example the refusal to use GitHub) and modern young musicians looking for a music typesetting solution (read this fascinating exchange).

10. No solution for the web

This point is closely related to the previous one, it really deserves being stated separately. Lilypond is a poor fit for usage on the web. While Lilypond can output SVG files, there’s no real tool that allows embedding Lilypond music snippets in web pages, there’s no support for responsive layouts, and there’s absolutely no way to interact with Lilypond from Javascript.

(Yes, I know there’s LilyBin, but it doesn’t allow embedding, and does not have an API).

11. Incomplete support for MusicXML

Lilypond comes bundled with a tool for importing MusicXML, but there’s no tool (to my knowledge, at least) for exporting to MusicXML.

12. Uncertain future

Music typesetting is enough of a niche market that any open-source endeavor will surely meet with difficulties maintaining momentum if it’s not able to garner enough enthusiasm among programmers. Compare for example, the current state of affairs in the Lilypond community to that of MuseScore, another open-source software backed by a commercial enterprise.

The sad fact is that the Lilypond community has not been strong or popular enough to be able to put together a foundation or some other entity that will ensure the future of this endeavor. Add to that the refusal to employ any partly-commercial online tools like GitHub, and no wonder some Lilypond bugs are 10-years old.

My feeling is that as the Lilypond project is aging (it’s now 20 years old), it’s community is becoming more and more isolated from the outside world, more conservative and resistant to change, less welcoming to new developers and quite frankly, not able to answer today’s needs in terms of music typesetting. After about 10 years of using Lilypond, I feel that investing any more effort in Lilypond is simply not worth it.

In the next installment, what’s wrong with commercial music typesetting solutions.

Michelle Obama Redefines the 2016 Election

In our hearts, we all know if we let Hillary’s opponent win this election, then we are sending a clear message to our kids that everything they’re seeing and hearing is perfectly OK. We are validating it. We are endorsing it. We’re telling our sons that it’s OK to humiliate women. We’re telling our daughters this is how they deserve to be treated. We’re telling all our kids that bigotry and bullying are perfectly acceptable in the leader of their country. (full transcript)

Michelle Obama has, with these words, redefined the 2016 presidential elections, she has reframed the issues at stake. It is no longer about politics, about programs. No, it is about who Americans wish to be, who they want to become: a narcissist bully, or a fighter for gender- and racial equality.

Why Did the Obamas Fail to Take On Corporate Agriculture?

But aspirations are cheap — and naïveté can be expensive. A few days after the candidate’s interview with Time, Senator Charles Grassley of Iowa, a.k.a. the Senior Senator from Corn, blasted Obama for his heretical views on American agriculture, suggesting he was blaming farmers for obesity and pollution. A campaign spokesman quickly walked back Obama’s remarks, explaining that the candidate was merely “paraphrasing an article he’d read.” Big Food had spoken, and the candidate — an urban politician from Chicago — got the first of what would turn out to be many unpleasant tutorials on the industry’s sway in Washington.

Meanwhile, the pace of 21st-century mass agriculture is unstoppable.