Book Production the Hard Way

Over the past few weeks I’ve been working to put a large project to bed. What started when a potential client contacted me about producing an ebook ended with the development of a combined ebook and PDF build system that saved a bunch of time when last minute changes were needed.

This project built on what I learned from a previous project where I paired an overly-complicated makefile with Pandoc to generate an ebook in French. This time, the project was in English which made it much easier to tailor my build system to the project. Since final page count of the PDF (6″×9″ trim size) came to just over 400 pages, I needed every shortcut and hack I could come up with to keep the ePub and PDF versions in sync.

Along the way I managed to have Pandoc, Make, Sed, Ruby, and LuaLaTeX all playing nicely together when I gave the command to make all.

Continue reading “Book Production the Hard Way”

MVW: The Concept of the Minimum Viable Writer

The MVW is a writer who can work in any situation while maintaining almost full productivity. The question becomes how to get there and what’s needed along the way?

I envision the MVW as someone who is hardware agnostic as possible. There’s no harm in having a favorite platform or writing app, but when the chips are down, the MVW can get by without.

Also lurking around the edges of the MVW idea is the question, why bother? We have all of this technology at our fingertips, and for the most part it’s not going anywhere. With a basic laptop as a working environment, it’s only a matter of closing the lid and stepping away from the desk to be mobile. With good backups, even the laptop becomes disposable. So why make all of these changes when things are working just fine?

In some ways, this might be considered much ado about nothing. In others, it could be the ultimate disaster recovery guide.

After my initial article outlining the idea of a MVW, I’ve had more time to think about the definitions and expand on them. Along the way I’ve clarified the concepts surrounding the idea.

Continue reading “MVW: The Concept of the Minimum Viable Writer”

Back on the Submission Hamster Wheel

Or back in the saddle again. Whichever you prefer. Regardless, I’ve started to submit my stories again. I put one in with Daily Science Fiction this morning. It’s 865 words of paranormal cop-show goodness. This feels like a turning point for me. One where I can get back to writing, with a better idea of the mechanics of putting a story together. I don’t know how big the slush pile at DSF is, but hopefully I’ll have a response soon enough.

Here’s the first line as a teaser:

Detective Jim Monrovia didn’t want to be standing in the Atlantic City dump looking at two bodies before he was properly caffeinated.

Blurb Magazine Template for Manga Studio

One of the neat things about Magna Studio EX (Clip Paint Studio EX if you buy the downloadable version) is it’s book making abilities. It shouldn’t be a surprise since the whole app revolves around making comics and comic books. The panel-based nature of comics also makes for a pretty neat layout tool for image heavy books or magazines.

In my case, it’s also a substitute for Adobe InDesign. For the occasional layout jobs I have, it’s not worth another subscription on top of Lightroom+Photoshop. So after fighting with Lightroom’s crippled book module1 I decided to build a template in Manga Studio that would work with Blurb’s magazine-sized books.

Continue reading “Blurb Magazine Template for Manga Studio”

Flash Fiction — The Escape

Ally tugged at the stethoscope around her neck and smoothed the stolen lab coat. The gray industrial carpet was hard under her feet. The tile floor of the crossing hallway was a few yards ahead. She looked at James. “Which way?”

In the hallway a few people walked by with morning coffee. The lingering caffeine in the air taunted them. James glanced at the emergency exit sign on the wall. The blue scrubs hung loosely on his shoulders as he pulled on a surgical cap. He pointed at the hallway. “There. Then left.”

They moved quickly into the hallway. She started to read the office signs as they walked. “203, 205, 207. This is it.”

James turned the handle and pushed open the steel door. The office had been gutted. The left half of the floor was missing. Yellow painted safety railing split the room. At the far end was a single flight of stairs leading to the floor below. Electrical service panels lined the walls. He looked over the railing, seeing nothing but an old wooden desk and double-doors in the wall below. She shouldered him into the room and closed the door. “Come on. The desk.”

They walked down the stairs, alert but moving fast. The desk looked like it was from the 1950s. All metal with two drawers on the left. A telephone on sat on the scratched desktop. He dug a crumpled piece of paper from a front pocket and punched in the number. He tapped his foot, waiting. He nodded when the other end picked up. “We’re here. Wait a sec.”

She pulled the old Timex from her bag and held it for a second. The metal case was cold and twitched with each tick of the second hand. She placed it gently on the desk, taking time to center it. They glanced at each other, both nodding. “Yeah. It’s ready,” he said into the phone.

A grid of red lasers swept over the desk. He looked up but couldn’t see where they came from. The lasers scanned over the desk, narrowing in on the watch. Once they found it, the grid tightened and started to spin around on the watch face.

She watched the red glow. “I feel like I’m at the self-check.”

He put his hand over the mouthpiece and whispered, “Hush.”

He ignored her glare. Then he rolled his eyes at the voice on the other end. “You happy now? I told you it was the real deal.”

After a second he hung up the phone. The lock on the desk clicked as the top drawer slid open slightly. Ally snatched the drawer open. They both paused, staring at the stacks of one hundred dollar bills. He looked at her with a smile. “I hope you brought the bag.” He noticed her shiver as she kept her eyes locked on the money. He let the moment pass. “Seriously, you did bring the bag, right?”

She snapped her eyes to his and let out a low laugh. She pulled a black nylon backpack from her shoulder bag. “Yeah, I did. Worry wart.” She pushed the bag at him and glanced at the upper level, “Get it loaded.”

There was a slight tremble in his fingers as he stuffed the stacks into the bag. He finished and tied a knot in the drawstring. He slung it over his shoulder and nodded towards the lower set of double doors, “Let’s go this way.”

She looked back at the desk, then at James with a raised eyebrow, “What about the watch?”

He slid it into his front pocket. With a slight twitch of his head he started moving towards the doors. She moved to catch up, and was along side of him when they reached the doors. She stopped and put her hand on the door’s push bar, “Why this way? That’s not the plan.”

“That hall up there is about to get real busy. The morning shift starts soon.”

Ally’s nostrils flared as she took a deep breath. She breathed out while pushing open the door. The cool air washed in as they stepped onto the polished concrete floor. The corridor had a low ceiling, and dim wire caged work lights along the wall. A slight breeze was blowing from the right.

He turned and started into the breeze. His voice was low, “This way.”

She lightly shoulder checked him, and pointed to the left. His mouth started to open and she shook her head. They locked eyes and she opened hers wide and nodded twice. She relaxed when the tension left his shoulders and he returned the nod. Halfway down the corridor she pointed at the door to the stairs. She opened it and he followed her into the stairwell. Two floors up, she stopped and nodded at the sign by the door.

“The maternity ward? How’d you know?”

She smiled. “The stairs on the fire exit sign by the elevator. I was hoping this connected.”

“Hoping? This is not the plan. We’re supposed to meet in the alley.” He was working to keep his voice low, and it came out in a hiss.

“You went off plan first by taking the lower hall.” She pointed at the bag. “We’re targets now. I want bright lights and middle-class wage slaves around me. Not druggies and hookers.”

He took a breath, “Our ride?”

“I’ll call her. We’ll meet a block over.” She pointed at the bag again, “Split it now.”

“What? Now?”

“Yeah, it’s a trust thing. So I don’t have to worry about you taking off.” She opened her shoulder bag and held it out to him. Her feet shifted into a fighting stance.


More on Ulysses and ePub

Ebook publishing seems to be my theme of the week. It’s something I didn’t know much about. I’ve managed to learn quite a bit this week. The most important thing I learned is about workflow.

There are lots of ways to make an ePub file. Amazon KDP will take just about any file and convert it to their .mobi format. Smashwords has a converter (“Meatgrinder”)that take a Word (.doc) file and do the conversion to ePub. These are convenient because they’re an all-in-one solution. But for the most part require a lot of work on the input file to avoid the garbage—in-garbage-out problem.

The Amazon and Smashwords guides are quite involved and require work to get the table of contents build in a way that will survive the Meatgrinder. Most other options are similar. Scrivener is another popular tool that will output an ePub directly. But work must be done inside the program to get the output correct.

Looking over the options, I still think I’m going to stick with Ulysses for making ePubs for now. It comes back to workflow.

I’m just one that would rather write in markdown, then export. This way all the formatting can be done through one stylesheet. From text to ePub is one click in the export panel. I can send the manuscript directly to iBooks for previewing. This is actually a huge time saver.

I’ve written a stylesheet for Ulysses that outputs clean formatting that will pass the KindleGen check and will convert to .mobi with no errors. This solved a lot of my problems with formatting. The default stylesheets contain CSS tags that get deleted during the conversion. It’s better to not have them in the file to start with. Once the new version, Ulysses for Mac (a name change from the current Ulysses III) ships, I’ll verify my stylesheet is still working and start a GitHub repository for it.

The exported ePub files will open in iBooks and convert to Kindle. I did find the files fail the ePub validation test1. This is because of the blank section problem I described in the last post. I also found out about a problem with the way the TOC file items are numbered. These are annoying, but easy enough to fix. In the end it’s easier to fix the files afterword than to deal with building TOCs by hand each time.

These bugs have been reported. Unfortunately, they won’t be fixed in time for the upcoming version.

Here’s what’s needed to fix an exported ePub file:

Start by putting the ePub file into its own folder. All the work will be done here. First, unzip the file.

$ unzip ulysses-fix.epub 

This will extract the contents, and leave the original file intact. The folder should have the following contents:

$ tree
│   └── container.xml
├── OPS
│   ├── book.opf
│   ├── cover.html
│   ├── cover.png
│   ├── css
│   │   └── style.css
│   ├── index.html
│   ├── navigation.html
│   └── toc.ncx
├── mimetype
└── ulysses-fix.epub

If there’s any comments before page headers, like this:

%% This is a comment
# This is a H1 header

Then jump back to my post on removing the blank leading section, and do those edits to the index.html, toc.ncx, and navigation.html files.

Once those edits are done, we can move on to fixing the “playOrder” bug. This is happens to all of the exports. If you don’t use leading comments, it’s the only thing that prevents ePub validation.

This one is a bit annoying, in that we’ll have to change several lines. The problem is that the playOrder items are not incremented. They are all stuck at 1. To pass validation, they need to count up with each TOC entry.

The sample file I’m using has one h1 header, two h2 headers, and one h3 header. Only the top two levels count for the TOC.

The toc.ncx file needs to be edited so that each playOrder line after the first gets incremented. Thes first line doesn’t need changing.

<navPoint class="h1" id="chapter-1" playOrder="1">

The following navPoint lines need editing.

<navPoint class="h2" id="section-1.1" playOrder="1">
<navPoint class="h2" id="section-1.2" playOrder="1">

The numbers need to increase so they are sequential. Like so:

<navPoint class="h2" id="section-1.1" playOrder="2">
<navPoint class="h2" id="section-1.2" playOrder="3">

This can be done by hand in a short TOC. With more than a few, it’s easier to automate. Check the bottom of this post for simple ruby script that will do this edit for you.

Once all the edits are done, we need to zip the file back up. This is best done with the -f option to zip, which will update (or “freshen”) the file with the changed files. Because we have the zip file and the extracted contents in the same folder, this command works:

$ zip -f ulysses-fix.epub 
freshening: OPS/index.html (deflated 54%)
freshening: OPS/navigation.html (deflated 51%)
freshening: OPS/toc.ncx (deflated 59%)

Now the same ePub file we started with will have the new files inserted into it. This is much easier that trying to re-zip the whole file. Now take the file to the validator and make sure it passes.

This will seem like a lot of work to start with. It’s really not, just annoying. The Ulysses developers know about the bugs. Soon these steps won’t be necessary. The benefits of one-click markdown > ePub export outweigh the hassle of having to edit the files for now.

Here’s the bonus ruby script for fixing the toc.ncx increment bug.

#!/usr/bin/env ruby
cnt = 1
ARGF.each_line do |line|
    if /playOrder/.match(line)
        line.sub!(/"1"/, "\"#{cnt}\"")
        cnt += 1
    puts line

Use it like this:

$ epub-fix.rb OPS/toc.ncx >

Inspect the file to make sure everything looks good, then replace the old one with it.

$ mv OPS/toc.ncx 

Then re-zip the file as shown above.

  1. To list a book on the iBooks store, an ePub file must pass validation with no warnings or errors.

Ulysses III export – ePub repair

This is what seems to be the second in a series about making ePub format eBooks with Ulysess III. Yesterday, I compared a few options for making eBooks. I found the least troublesome way was to just simply export from Ulysses using a custom stylesheet. Today’s post is about how to fix an annoying export bug.

Ulysses is a three-paned editor. The second pane is a list of sheets in the current working folder. The first part of the first line of any sheet is shown there. The sheet can be sorted by name, using this first line. But what if the sheets in a longer project don’t have titles that are alphabetical?

This is where a note can keep things organized. A quick note at the top of a sheet with a leading number keeps everything in order. There’s an option to sort by a custom order, but it applies to all folders. This is why I override it for certain projects.

A lot of my sheets start with something like this:

%% 01 Short description
# The Heading

This one is going to sort to the top, just where I want it. But there’s one slight problem. It breaks things.

A comment—in either block (%%) or inline (++) style—at the top of a sheet gets translated to an empty paragraph. This isn’t a problem with HTML. It’s just a extra carriage return at the top of the file.

With ePub export it’s much more of a problem. There’s going to be a blank line in at the top of the contents. And depending on the stylesheet used, maybe even a blank starting page. What’s happening is that the empty paragraph is being converted into an empty section. Which in ePub formatting is an empty chapter.

The following are examples from iBooks. I made up a quick sample using the excellent Lorem Gibson text generator.

The default stylesheet starts with a blank page.
A custom stylesheet starts the book on the left page.

If you export from Ulysses using one of the included templates, there’s a blank page at the front. In iBooks this means the start of the book in on the right page in a two page layout. With a custom stylesheet, the books starts on the left but the blank section is still there. Looking at the table of contents you’ll see a blank line at the top. This is a link to the empty section.

This only a cosmetic annoyance when reading the book in iBooks. The problem comes when using the KindleGen file converter to make .mobi files. It chokes on the blank section and won’t do the conversion.

The solution

The easy way it to delete any comments that come before the first heading. This is the best way to go in most cases. But if you really need to keep sheets in order, it’s not so great.

If you keep comments in, and export to ePub, there’s only one way to fix it. You have to open the ePub file and edit the contents. Ebook surgery isn’t hard, and I’ll walk you though it.

First we need to unpack the file. ePub is just a zip archive with an .epub extension instead of .zip. There’s a couple of ways to do this. The command line unzip command works, along with the free The Unarchiver app.

Start by creating a new folder, and move your ePub file into it. Then, right click and Open with > Other… and choose The Unarchiver. From the command line do unzip <filename>. Now you’ll have a folder with the following contents:

$ tree
│   └── container.xml
├── OPS
│   ├── book.opf
│   ├── cover.html
│   ├── css
│   │   └── style.css
│   ├── index.html
│   ├── navigation.html
│   └── toc.ncx
└── mimetype

The files that need editing are:

  • index.html
  • toc.ncx
  • navigation.html

For index.html delete the highlighted portion.

index.html showing empty section. Click to enlarge.

For toc.ncx delete the highlighted portion.

toc.ncx showing empty navpoint. Click to enlarge.

For navigation.html delete the highlighted portion.

navigation.html showing empty list item. Click to enlarge.

Those are the artifacts caused by the comment at the top of the sheet. Once they’re gone the ePub file will behave. It will also now covert to a Kindle .mobi file.

But wait, how does a folder of files get turned back into an ePub?

I’m glad you asked!

It’s simple. Just zip the files up and change the .zip extension to .epub!

Here’s the easy way:

  • In the Finder, select the three top level items: META-INF, OPS, and mimetype.
  • Right-click and choose “Compress 3 items”
  • Change the name to My Great Ebook.epub1.

See, I told you an .ePub file is just a zip file. Now your file should be fixed and work correctly.

As I write this, the makers of Ulysses have a new version that should come out soon. Hopefully this bug will be fixed.

  1. Or your real title. Either way I won’t judge.