Notes on publishing ebooks, including scripts
Take this as analogous to the blind men describing an elephant. I’m far from a best-selling author and have used only Lulu.com, Amazon’s Kindle Direct, and Smashwords. I’m sure there are many things I didn’t run into or didn’t even think to look for. I’ll use Amazon (Kindle Direct Publishing), Lulu, and Smashwords as examples because they are the services I currently use. The field is still rapidly developing—Smashwords has just been merged into Draft2Digital—and what I chose two years ago might not be what I would choose today.
For example, while it normally irks me when a service I choose gets bought up by one I didn’t choose (AT&T, I’m looking at you), I’m not feeling too badly about the Draft2Digital purchase of Smashwords. When I started writing this post, I wrote that if I were to start again, I’d be seriously looking at Draft2Digital instead of Smashwords.
In any case, one way to find such places is to search for “Smashwords competitors”, or, now, “Draft2Digital competitors”.
And also upfront: this is about the technical end of getting things online. It’s not about how to sell or how to design. You wouldn’t want to listen to me about those topics anyway. I could just as well have posted this under hacks instead of under books. This is a lot about the command-line workflows I use to get a book from Nisus (or any word processor) to ePub. There is math involved, or at least programming. I’ll also provide some of the scripts I use, but you’ll need to be comfortable with modifying them, or have friends who are comfortable modifying them, for your own special circumstances. If there’s one thing I’ve learned from Smashwords, it is that every book is different in its own annoying way.
Currently, I use Lulu or Amazon for print-only books—mainly, books that I’m releasing free as ebooks, such as my fantasy roleplaying game or my computer manuals. I’ll use my web site to host the ebooks and make print copies available through Lulu or Amazon. Amazon, of course, is the default place to go for buying novels, but Lulu.com is extremely useful for niche products where the community buying those products is familiar with Lulu.com. That’s why my game books are there.
For the most part, when my plan is to sell a print book mainly online, Amazon is the place to go. If my plan were to sell a print book everywhere, I’d probably want to find somewhere else rather than use Amazon as the bottleneck. But I would not pay for the service. Payment would be for the books themselves, as needed.
Unlike Amazon and Lulu, the purpose of an e-book distributor such as Smashwords is not to sell it on the distributor’s site, but to distribute it to other sites.1 Smashwords, for example, distributes to Apple, Barnes & Noble, and several other e-book sites. The advantage to the reader of buying on those other sites is that the reader can take advantage of the features those other apps provide. If your customer reads ebooks in Apple’s Books app, and buys their books from the Apple bookstore, updates to books come to their device automatically. This is especially important for instruction manuals, such as a coding book, or safety manuals, or other vaguely textbook-like things.
And as a side note that deserves more emphasis than a mere footnote, do not choose an ebook distributor that you are not comfortable updating after publication. Your book is filled with errors, at least one of which is very consequential. When you discover that error, fix it and re-upload. If the process used by an ebook distributor makes regular updates difficult, do not use that distributor.
If you don’t update your book at least once after making it available for sale, you aren’t taking advantage of the benefits that ebooks and print on demand provide the independent writer.
Most e-book distributors have a conversion process for converting Microsoft Word documents to an ebook format (Kindle for Amazon, and ePub for everyone else). Open Office, and probably most word processors, such as my own favorite Nisus Writer, can save as an RTF file in Word format if you happen not to be using Word. The distributor’s automatic conversion is useful only for very simple books—novels, mainly, because that’s where the market is for ebook distributors—but it is useful. It means, for example, that you can make updates in your word processor, re-upload the word processor file, and let the distributor convert the new update for redistribution.
Whatever distributor you use should have some sort of style guidelines for what the document should contain and what the automatic convertor can and cannot handle.
If you want your book available other than on the distributor’s site, you’ll need an ISBN. The purpose of an ISBN is to (a) uniquely identify a book for purchasing, and (b) direct the seller (the bookstore or library) to the correct publisher. If you buy your own ISBN, this will allow you to keep that ISBN regardless of which printer or ebook distributor you use. However, it also means that you have the responsibility of directing the purchaser or seller to whoever you happen to be using to print or distribute the book.
By using an ISBN from the printer (such as Amazon) or from the distributor (such as Smashwords) potential sellers will be directed automatically to that printer or distributor for purchases. This is by far the easiest choice, but it does mean locking yourself into that distributor for that version of the book. If you change distributors, you’ll change ISBNs.
Example 1: The Dream of Poor Bazin
Versions: Print on Amazon, Kindle on Amazon, ePub etc. through Smashwords to Apple, etc.
The Dream of Poor Bazin is a novel. From a publishing standpoint, this is the simplest kind of book to do. The publishing services market is oriented almost exclusively to novelists, because they vastly outnumber everyone else. I uploaded a PDF to Amazon for the print version, and then uploaded an RTF file to Amazon for the Kindle version and to Smashwords for the ePub version (and other obscure formats) for distribution to Apple, Barnes & Noble, and lesser outlets. Apple, Smashwords, and Barnes & Noble make up 100% of my sales across all three books I have on Smashwords, in that order.2
There are no images in the novel. The cover is a PDF uploaded to Amazon, and a PNG uploaded to Smashwords. Any PDF cover you upload must have the fonts outlined. This is another way of saying that the PDF for your cover must not have any embedded fonts; you cannot rely on the distributor using those fonts correctly in a wraparound cover.3 I use Scribus to create my covers, so I choose “Outline all fonts” for the “Embedding mode”. “Outlining” means converting the letters of the cover into (vector) graphics, so that it doesn’t confuse the cover printer.
I created my cover using Inkscape, GIMP, and Scribus. Scribus is the final part of the cover-creation chain, and is the app that must outline fonts.
The distributor’s guidelines will tell you the size needed for the cover of an ebook. Covers for the print book are different, because they need to wrap around, and the spine’s width changes depending on how many pages the book has. For this reason, I have a command-line script for the Mac that will calculate the wraparound cover size required for a print book at Amazon. It’s designed solely for me, and so is not the easiest to read, but if you want a copy of the script you’re welcome to it (Zip file, 1.1 KB). I don’t need this for Lulu because their wraparound cover upload process says exactly what size the cover should be.
Example 2: 42 Astoundingly Useful Scripts and Automations for the Macintosh
Versions: Print on Amazon, custom ePub through Smashwords to Apple, etc.
It probably wasn’t the best idea to make the most complicated book be the second one I tried. I justified it by saying that if I could publish a book of command-line scripts for the Macintosh as an ebook, I could publish anything as an ebook. This book contains tables, footnotes, multiple fonts, and images. The first two are not supported at all by the Smashwords converter (or, at the time, by any distributor I could find), and multiple fonts and images are only poorly supported at best. Even now, whenever I upload a new version to Smashwords, there’s about a one-in-three chance that Smashwords will flag it for some image issue that’s not really been an issue for the last decade, such as images being wider than 600 pixels.4
The ePub format is literally just a limited version of HTML and CSS, wrapped up in a zip file. What I ended up doing was creating this custom HTML myself, with a combination of custom scripts and a program called Pandoc.
- I convert the document to HTML using a Nisus Writer Pro script (Zip file, 14.7 KB).
- I convert the HTML to ePub-friendly HTML using a custom Perl script (Zip file, 3.1 KB).
- I replace the quality images in the HTML with Smashwords-friendly images using a script.
- I use Pandoc to convert the HTML files to an ePub zip file using my custom CSS and metadata file.
- I unzip the ePub file to fix the footnotes and the chapter titles in Pandoc’s HTML.
- I rezip the HTML back into an ePub file.
- I run a program called epubcheck on the file to make sure it has no problems.
Of course I have a script that combines steps 2 through 7. Again, except for Pandoc and epubcheck, which I did not write, these scripts are heavily oriented to my own needs, but if you are or have a programmer who wants to look at them, here they are (Zip file, 3.1 KB).
The metadata file is an XML file; mine looks like this:
[toggle code]
-
- <dc:language>en-US</dc:language>
- <dc:creator opf:file-as="Stratton, Jerry" opf:role="aut">Jerry Stratton</dc:creator>
- <dc:publisher>Jerry Stratton</dc:publisher>
- <dc:date opf:event="publication">2020-08-15</dc:date>
- <dc:rights>Copyright ©2020 Jerry Stratton</dc:rights>
- <dc:description>Get your retro on with 42 Astoundingly Useful Scripts and Automations for the Macintosh. These modern scripts will help you work faster and more reliably, and inspire your own custom scripts.</dc:description>
- <meta property="ibooks:version">2.0.1</meta>
- <!-- These do not get included; note that those specific ids are required to even get the titles included -->
- <meta refines="#epub-title-1" property="title-type">main</meta>
- <meta refines="#epub-title-2" property="title-type">short</meta>
The reason I convert to HTML first is to create a completely clean document that is easily styled. The Nisus Writer script I use goes through chunk by chunk and converts word processing styles into <span>
and <div>
tags. Any formatting that is not a style, I throw out as not for anything except print. This creates very clean HTML that is easily styled using CSS. Because ePub is nothing more than a limited form of HTML, this makes a custom conversion to ePub much simpler.
The CSS file applies special formatting to various classes; here are some of the CSS definitions from my CSS file as an example:
[toggle code]
- /* Astounding Scripts CSS */
-
body {
- font-family: Adobe Garamond, Georgia;
- }
- /* titles */
-
h1 {
- margin-top: 0;
- text-transform: uppercase;
- font-weight: bold;
- font-size: 172%;
- padding-top: 1em;
- padding-bottom: 1em;
- text-align: center;
- }
-
h2, h3 {
- page-break-before: always;
- margin: 0;
- padding: 0;
- margin-top: 1em;
- margin-bottom: .5em;
- text-align: center;
- }
- /* paragraphs */
-
p {
- text-indent: 1em;
- text-align: justify;
- margin: 0;
- }
-
div.first_line p {
- margin-top: .5em;
- text-indent: 0;
- }
- /* code */
-
div.computer_output, div.code_printout, span.inline_code {
- margin-top: 1em;
- font-family: monospace;
- }
Example 3: The Missing Indices
Versions: Print on Amazon, free PDF on web site
It turns out not to be true that if I could publish 42 Astounding Scripts as an ebook, I could publish anything. From a publishing standpoint, the worst kind of book to make an ebook of is an index, and both The Missing Index For the Cookbooks of the Saint Mary’s Altar Society of St. Michael’s, Brunswick, and Christ the King, Hesperia and The Missing Index for the Southern Living Cookbook Library are literally all index. Neither Amazon’s Kindle format converter nor Smashwords’s ePub converter handle tabbed lines. Fortunately, I wasn’t planning on selling this, I was giving it away for free, and to a very small audience. So I put the PDFs on my web site for free, and then uploaded the PDFs to Amazon for the print copy, along with a wraparound PDF cover.
There are some books that ebook publishers simply cannot handle well, and indexes are one of them. I could use the same process I used for 42 Astounding Scripts to create a custom ePub file for these books—ePub, after all, is HTML and CSS, and HTML/CSS can easily handle index tables—but it isn’t worth the reward. These are very niche products, and most of that niche is going to want a print copy anyway.
Example 4: SuperBASIC: For the TRS-80 Color Computer
Versions: Print on Amazon, free ePub through Smashwords to Apple, etc.
This is a manual for the SuperBASIC script I wrote for a retro 8-bit computer, the TRS-80 Color Computer. For the most part, this was not a difficult document to convert to ebook format, except that as a manual it has tables to summarize various parts of the program. Smashwords doesn’t support any form of tabbed data, explicitly including tables. So for the ebook version I wrote a script for Nisus Writer Pro that converts tables to images. This is not optimal because it means that the content of tables are neither searchable nor readable by text-to-audio tools. But since the content of the tables is also repeated, more wordily, in the text, I decided this wasn’t a big enough issue for me to go to the trouble of creating a custom ePub file.
I put the table-to-image script online at Text to image filter for Smashwords conversions.
I chose to distribute the SuperBASIC manual on Smashwords so that programmers who buy the ebook will get automatic updates when I update SuperBASIC’s functionality.
Example 5: The Gods & Monsters RPG rulebooks
Versions: Print on Lulu.com, PDFs on web site
Much like Amazon, Lulu.com allows you to upload a PDF and people can then purchase that as a print-on-demand paperback (or hardcover, but they’re expensive) from Lulu.com. For the Gods & Monsters rulebooks it works because these are hobby books for a hobby audience that is already used to going to Lulu.com for print copies of books. The main drawback is the lack of free shipping. This makes Lulu books a bit more expensive than Amazon books. There is no “free shipping over $x” on Lulu.com.
I used Lulu instead of Amazon because Amazon’s CreateSpace (since replaced by Kindle Direct Publishing) wasn’t a viable option at the time—I started publishing these on Lulu back in 2004. I continue to use Lulu instead of Amazon because Lulu has a larger number of formats, including the 9x7 format that I like for portable rulebooks.
Like Amazon, a PDF cover file destined for Lulu must not embed any fonts; they must be outlined.5 Like Amazon, the cover is uploaded as a separate wraparound image, with the spine size varying depending on the number of pages. Lulu has a better interface than Amazon, however, and tells you exactly what size the wraparound image should be and where the spine starts and ends.
You can pay to have your Lulu.com book distributed on Amazon, but there’s no point: you can use the same PDF directly on Amazon for free. Assuming that Amazon has the same format size that you used on Lulu, of course, but if you don’t, I’m betting your book is niche enough that the extra cost isn’t worth it.
That is a very critical point: Lulu or sites like Lulu will not be useful to you unless your audience is limited to people that will go to Lulu.com to purchase the book. That works for RPG books, because the indie RPG community is a community on Lulu. But I don’t expect that’s true for more general books, or for all niche communities.
That can be the purpose of a printer, as well, but doesn’t have to be, and unless you’re a huge seller, probably shouldn’t be what you count on. If you choose Amazon, it is because your market is comfortable buying from Amazon. If you choose Lulu, it is because your market is comfortable buying from Lulu. Any other sales are gravy.
↑Of course, my sales are likely tilted toward Apple because one of my books—42 Astoundingly Useful Scripts and Automations—is about programming on the Macintosh.
↑No, I don’t know why embedded fonts are unusable in cover files but not in the content itself. I’m afraid to question it too much, because then embedded fonts will start failing in the content, too.
↑While it is true that 42 Astounding Scripts is an homage to the BASIC compilations of several decades past, that doesn’t mean I want the image quality from several decades past.
↑Also like Amazon, Lulu handles—and in fact requires—embedded fonts in the document file. But your “save as PDF” in your word processor should embed fonts automatically.
↑
books
- 42 Astoundingly Useful Scripts and Automations for the Macintosh
- MacOS uses Perl, Python, AppleScript, and Automator and you can write scripts in all of these. Build a talking alarm. Roll dice. Preflight your social media comments. Play music and create ASCII art. Get your retro on and bring your Macintosh into the world of tomorrow with 42 Astoundingly Useful Scripts and Automations for the Macintosh!
- The Dream of Poor Bazin (Official Site)
- The Dream of Poor Bazin: A Novel of adventure journalism in Washington, DC.
- Gods & Monsters Rules
- All rules on creating characters and playing the game. This is what you’ll be using most as both player and adventure guide.
- A manual for superBASIC
- I’ve improved superBASIC and also made a short manual available on Amazon and most e-bookstores.
- The Missing Index for the Southern Living Cookbook Library
- I’ve compiled an index for all 22 (I think) of the Southern Living Cookbook collection. You can download it here, or buy a print copy if, like me, you prefer browsing.
- St. Mary’s Altar Society Cookbooks
- The missing index for the St. Mary’s Altar Society cookbooks from 1976 through 1981.
publishing services
- Lulu, Nisus, and Gods & Monsters
- Lulu is sometimes really annoying. But they usually get the job done. Nisus, on the other hand, is rarely annoying to use and always gets the job done.
- Lulu.com
- Self-publishing services with no upfront fee. I’ve had good experiences with them on Gods & Monsters and as a sort of “private printer”.
- Smashwords
- “Your ebook. Your way.”
scripts
- ePub conversion helper scripts (Zip file, 3.1 KB)
- Scripts to help when using Pandoc and epubcheck.
- EPUBCheck: The conformance checker for EPUB publications
- “EPUBCheck is a command line tool. EPUBCheck evaluates EPUB publications against the official EPUB specifications. It supports both EPUB 2 and EPUB 3.”
- KDP Cover Calculator (Zip file, 1.1 KB)
- A Python script to calculate the width of a wraparound cover for an Amazon print book.
- Nisus HTML Conversion scripts (Zip file, 14.7 KB)
- This script will convert a simple Nisus document (no tables or lists) into very clean HTML.
- Pandoc: a universal document converter
- “If you need to convert files from one markup format into another, pandoc is your swiss-army knife.”
- Text to image filter for Smashwords conversions
- Smashwords has very strange requirements for ebooks. This script is what I use to convert books to .doc format for Smashwords, including converting tables to images.
writing tools
- GIMP
- The GIMP image manipulation program is one of the best free software packages of any kind. Fully multi-platform, the GIMP “is suitable for a variety of image manipulation tasks, including photo retouching, image composition, and image construction”.
- Inkscape
- “Inkscape is an Open Source vector graphics editor. Supported features include shapes, paths, text, markers, clones, alpha blending, transforms, gradients, patterns, and grouping. Inkscape also supports Creative Commons meta-data, node editing, layers, complex path operations, bitmap tracing, text-on-path, flowed text, direct XML editing, and more. It imports formats such as JPEG, PNG, TIFF, and others and exports PNG as well as multiple vector-based formats.”
- Nisus Writer Pro 2.0
- The new Nisus is pure awesome: very easy to use, and it does everything I need.
- Scribus
- Scribus is a very nice open source page layout application and includes full PDF creation. It is also scriptable using Python if you need to automate page layout tasks. Scribus is very useful for making documents that need to be shared with other editors, since anyone can get the Scribus application unrestricted.
More Amazon
- California threatens Amazon, kills affiliate programs
- By this time, California had to know that its new law would not bring in new tax revenue. The tax headaches aren’t worth the trouble of maintaining affiliate programs. The only reason to pass the law was to kill affiliate programs at places like Amazon and Overstock. I don’t understand; what is it about affiliate programs that states don’t like?
- Regulations cost money
- Regulations cost money. Do politicians really not understand this?
- George Orwell’s incinerator
- Amazon shows by doing why digital restriction management on consumer items is a bad idea.
- Why does Amazon get the links?
- Why, when I love independent games, independent music, independent movies, independent comics, and independent books, do I link to Amazon.com for all of those?
More ebooks
- Eucalyptus development ends, removed from app store
- Jamie Montgomery has stopped selling the Eucalyptus Gutenberg reader.
- Publishers hoist on their own etard
- The publishers who are complaining about Amazon’s lending library appear to have screwed themselves.
More self-publishing
- Self-publishing for the masses
- Self-publishing is becoming easier and easier nearly every day. Successful self-publishing remains just as difficult, if it isn’t getting even more difficult because of all of the new self-published books on the market. The combination of print-on-demand and on-line bookstores has revolutionized self-publishing in the same way that the world-wide web has revolutionized having an opinion.
More Smashwords
- Text to image filter for Smashwords conversions
- Smashwords has very strange requirements for ebooks. This script is what I use to convert books to .doc format for Smashwords, including converting tables to images.