Mimsy Were the Borogoves

Hacks: Articles about programming in Python, Perl, Swift, BASIC, and whatever else I happen to feel like hacking at.

Text to image filter for Smashwords conversions

Jerry Stratton, May 13, 2020

Gettysburg Address: Lincoln’s Gettysburg address, created with text2image.; Abraham Lincoln; Civil War

Smashwords has a lot going for it, but it also has a lot of weird rules for what can go into books they publish, especially what can go into books that go through their automated conversion process. You’ll definitely want to read the Smashwords Style Guide before running any manuscript through their system. Some of the requirements are very counterintuitive even if—in some cases especially if—you are already familiar with ePub and other ebook formats.

Smashwords is mainly geared toward books with only two kinds of text in them, chapter titles and paragraphs—basically, novels. It was pretty much impossible to get a book of code to convert reliably with their automated process, so I ended up using Nisus Writer Pro’s macros and a couple of command-line scripts to create a custom ePub file for 42 Astoundingly Useful Scripts and Automations for the Macintosh.

By creating the ePub by hand I was able to carefully massage the book’s content to allow easily copying the scripts from the book so that people don’t have to type code if they don’t want to.1 This also meant, however, that 42 Astounding Scripts is only available in ePub format on Smashwords.

When I decided to write a manual for superBASIC it seemed it ought to be simple enough to go through the automated process and so take advantage of the multiple formats. SuperBASIC is a tool that allows writing old-school code using modern structures, but it’s still BASIC.

I still ran into a couple of issues but was able to script them away using Nisus Writer Pro’s easy and powerful macro language. Most of it was pretty simple, as you can see in the AppleScript I used to invoke the macro code:

[toggle code]

  • -- convert Nisus document for use on SmashWords
  • -- Jerry Stratton astoundingscripts.com
  • tell application "Nisus Writer Pro"
    • --get path to save table images
    • copy path of document 1 to documentPath
    • tell application "Finder"
      • set documentFolder to parent of (POSIX file documentPath as alias)
      • set documentFolder to POSIX path of (documentFolder as string)
      • set smashwordImageFolder to documentFolder & "/Book Graphics/Smashwords"
    • end tell
    • set imageFile to documentFolder & "temporary.png"
    • --convert tabs and properties
    • Do Menu Macro with macro "
      • $original = Document.active
      • $smashwords = Document.newWithText $original.text
      • #convert preceding tabs to hard spaces
      • Replace All \" \", \"    \"
      • #convert properties
      • Select All
      • Convert to Fixed Content
    • "
    • --replace with low quality images, 500 pixel max width
    • Do Menu Macro with macro "
      • Select Document Start
      • #$document = Document.active
      • #$images = $document.allImages
      • $smashwordImageFolder = " & quoted form of smashwordImageFolder & "
      • File.requireAccessAtPath $smashwordImageFolder
      • While Select Next Image
        • $image = Image.selectedImage
        • $imageName = $image.fileName
        • $replacementFile = $smashwordImageFolder.filePathByAppendingComponent $imageName
        • $replacementImage = Image.newFromFileAtPath $replacementFile
        • Image.insertInlineImage $replacementImage
        • Format:Paragraph Style:Image Line
      • End
    • "
    • --convert tables to images
    • repeat
      • Do Menu Macro with macro "
        • Select Document Start
        • $document = Document.active
        • $tables = $document.text.tables
        • If $tables.count > 0
          • $tableSelection = TableSelection.new $tables[0]
          • $document.setSelection($tableSelection)
          • Convert Table to Text
        • End
      • "
      • copy selected text of document 1 to tableText
      • if tableText is "" then exit repeat
      • repeat until last character of tableText is not "
  • "
        • set tableText to (characters 1 thru -2 of tableText) as string
      • end repeat
      • --convert table text to image
      • --delete any existing image
      • tell application "Finder"
        • if exists imageFile as POSIX file then
          • delete imageFile as POSIX file
        • end if
      • end tell
      • --convert table text
      • set tableText to quoted form of tableText
      • set quotedImage to quoted form of imageFile
      • set converterScript to "echo " & tableText & "| ~/bin/text2image --fgcolor .5,.5,.5 --border --restrict 500 " & quotedImage
      • set imageResponse to do shell script converterScript
      • tell application "Finder"
        • if not (exists imageFile as POSIX file) then
          • display dialog "Not successful creating image: " & imageResponse
          • return
        • end if
      • end tell
      • --insert image into document
      • Do Menu Macro with macro "
        • File.requireAccessAtPath " & quoted form of documentFolder & "
        • $tableImage = Image.newFromFileAtPath " & quotedImage & "
        • Image.insertInlineImage $tableImage
        • $newWidth = 200
        • $newHeight = ($tableImage.height*$newWidth)/$tableImage.width
        • $tableImage.height = $newHeight
        • $tableImage.width = $newWidth
        • Format:Paragraph Style:Image Line
      • "
    • end repeat
    • --export to Word format
    • Do Menu Macro with macro "File:Export As…"
  • end tell

The macro code removes all tabs, converting them to hard spaces; it converts automatic content, such as last saved date and document properties, to fixed content, and then it prompts to save the result as a Microsoft Word .doc file.

Augmented assignent operators in superBASIC: Table of the augmented assignment operators in superBASIC. Table created with text2image Swift script.; programming; mathematics; superBASIC

Tables are critical to a good manual.

The one big problem is that Smashwords’s system absolutely forbids tables. It’s difficult to write a manual without tables. Good, simple tables are critical to a great manual. They provide information at a glance that readers need quickly, and organize it in a manner that make that information easier to remember.

Smashwords’s recommendation is to convert tables into images. Nisus has no facility to automatically Convert Table to Image, for the obvious reason that this is nuts, but having to recreate tables by hand every time a manual is updated means that updates are going to be neglected. So I looked into a means of piping tab-delimited data to a command-line script. First, I tried using ImageMagick’s convert command, and it did work, after a fashion.

[toggle code]

  • #!/usr/bin/perl
  • #convert text to PNG image
  • # Jerry Stratton astoundingscripts.com
  • $command = '/opt/local/bin/convert -size 1000x2000 xc:white -font "Courier" -pointsize 12 -fill black -annotate +15+15 "@-" -trim -bordercolor "#FFF" +repage png:-';
  • open($converter, "|$command|");
  • while (<>) {
    • print $converter $_;
  • }
  • $image = <$converter>;
  • print $image;

The quality of the image was pretty bad, and it required using a monospaced font. Convert doesn’t handle tabs, which meant replacing tabs with spaces2, which in turn meant that the characters needed to be monospaced or the columns wouldn’t line up.

Another option would have been to do table-to-text, run it through something like the aligntabs script from 42 Astounding Scripts, and then apply a monospaced font to the text. This didn’t work well in tests. The converter didn’t always detect the monospaced font. In any case this would have produced unreadable tables on ebook readers that allow resizing screens, or that wrap on small screens, because part of the point of tables is that if and when they wrap, the columns remain columns.

But as it turns out I already had a script for generating images from text using macOS’s own high quality font and image creation libraries: the asciiART script from 42 Astounding Scripts. The tables I need to convert are very simple, one-line-per-row. It seemed very likely that I could write a script to convert the tables to text and then the tabbed text into a quality PNG.

The result is a very simple Swift script. It reads text from the standard input, which means anything piped to it.3 The text is placed in an NSMutableAttributedString; each line gets a newline except the final one—otherwise, the image would have a bunch of blank space at the bottom.

Butter and pecan recipes from Food & Wine: A table of recipes using pecan and butter, from Food & Wine’s annual cookbooks, created using a custom database piped to a Swift text-to-image converter.; Food & Wine Magazine; pecans; butter

The script automatically sets tab stops according to the maximum width of each column’s contents.

If there are tabs in the text, the script will attempt to calculate appropriate column widths, and then set tab stops accordingly. The spacing between tab stops is proportional to the font size. For example, I have a script that queries a custom Food & Wine recipe database, and outputs the year a recipe appeared and its page number in that year’s annual cookbook. It’s a tab-delimited list perfect for piping to text2image:

  • foodwine butter pecan | sort | text2image pecan.png --font "Adobe Garamond" 24

Notice that the script attempts to align columns appropriately. Numeric columns are aligned right, and everything else left.4

It will wrap long lines if you specify --wrap pixels on the command line. It determines how much space the text requires by using NSAttributedString’s boundingRect method. By providing a width but no height—that is, setting height to zero—macOS calculates the height needed for the text at that width.

It can also put a thin border around the text, and if it does, it pads out the text to make space for it.

Since it’s just text with a potential hairline border, increasing the dimensions of the image is as simple as increasing the font size:

  • pbpaste | text2image Gettysburg.png --wrap --border --font "Adobe Garamond" 48

As you can see in the image on this page, increasing the font size without anything else for the font size to be relative to just means increasing the resolution of the image. You can specify a specific pixel width that you want the image to not exceed, or you can let it default to about fifty character widths5, as above. If you do provide a width, it must be the first option after --wrap.

When specifying the wrap, you can also specify whether paragraphs are indented, double-spaced, fully justified, or flush right. If they’re indented, the script ignores blank lines.

Because Smashwords recommends a maximum image width of 500 pixels, the script also allows restricting the width; if the resulting image exceeds the width specified with --restrict, it resizes the image to the requested maximum.

Finally, the script converts the image to a bitmap and saves it as either TIFF, JPEG, or PNG, depending on the suffix you’ve specified for the filename.

This script is why I’m using an AppleScript to do the conversion process instead of doing it natively in Nisus Writer Pro’s wonderful macro language. Mac applications are sandboxed, and while there is a way out of the sandbox for reading and writing files, there is no way out of the sandbox for running command-line commands such as any shell scripting language.6

AppleScript, however, can, using do shell script. Thus the somewhat convoluted way that the AppleScript loops through each table, converts it to text, copies the text to a variable, and then pipes the contents of that variable through text2image. This is all to get around the sandbox by using an AppleScript that runs outside the sandbox.

[toggle code]

  • #!/usr/bin/swift
  • //convert text from standard input to image file
  • // Jerry Stratton astoundingscripts.com
  • import AppKit
  • //command-line arguments
  • var arguments = CommandLine.arguments
  • var commandName = arguments.removeFirst()
  • func help(message: String = "") {
    • if message != "" {
      • print(message)
    • }
    • print("Syntax:", commandName, "[--options] <imagefile>")
    • print("\t--help: print this help and exit")
    • print("\t--bgcolor <r,g,b,[f]>: background color for image file")
    • print("\t--border: draw a border around the image")
    • print("\t--fgcolor <r,g,b,[f]>: foreground color for image file")
    • print("\t--font <name [size]>: font name and optionally size for image file")
    • print("\t--restrict <width>: restrict image to maximum width")
    • print("\t--wrap <x> [justify right center indent double]: maximum width of image, in pixels, for wrapping text with paragraph options")
    • print("\timagefile: name of image to create; must end in .png, .jpg, or .tiff")
    • exit(0)
  • }
  • //image manipulation routines
  • func scaleImage(image:NSImage, width:CGFloat) -> NSImage {
    • //calculate new height from width
    • let height = image.size.height*width/image.size.width
    • let scaledSize = NSSize(width: Int(width), height: Int(height))
    • //resize image
    • let scaledImage = NSImage(size:scaledSize, flipped: false) { (resizedRect) -> Bool in
      • image.draw(in: resizedRect)
      • return true
    • }
    • return scaledImage
  • }
  • //parse arguments
  • func stringToColor(color:String) -> NSColor {
    • let parts = color.components(separatedBy: ",")
    • var rgbf = parts.map { CGFloat(($0 as NSString).floatValue) }
    • if rgbf.count < 4 {
      • rgbf.append(CGFloat(1.0))
    • }
    • return NSColor(red:rgbf[0], green:rgbf[1], blue:rgbf[2], alpha:rgbf[3])
  • }
  • func popFloat(warning: String = "", defaultValue: CGFloat? = nil) -> CGFloat {
    • var value:CGFloat = 0.0
    • if arguments.count > 0 {
      • value = CGFloat(Float(arguments[0]) ?? 0)
    • }
    • if value == 0 {
      • if defaultValue != nil {
        • value = defaultValue!
      • } else {
        • help(message:warning)
      • }
    • } else {
      • arguments.removeFirst()
    • }
    • return value
  • }
  • func popString(warning: String) -> String {
    • var value:String = ""
    • if arguments.count > 0 {
      • value = arguments.removeFirst()
    • } else {
      • help(message:warning)
    • }
    • return value
  • }
  • var imageFile = ""
  • var borderWidth:CGFloat = 0.0
  • var fontSize:CGFloat = 24.0
  • var indent = false
  • var hyphenate = false
  • var doublespace = false
  • var alignment = NSTextAlignment.left
  • var backgroundColor = NSColor(red:1, green:1, blue:1, alpha:0)
  • var foregroundColor = NSColor.black
  • var fontAttributes = [
    • NSAttributedString.Key.font:NSFont(name: "Times New Roman", size: fontSize)!,
    • NSAttributedString.Key.backgroundColor: backgroundColor,
    • NSAttributedString.Key.foregroundColor: foregroundColor,
  • ]
  • var maximumWidth:CGFloat = 0.0
  • var restrictedWidth:CGFloat = 0.0
  • while arguments.count > 0 {
    • let argument = arguments.removeFirst()
    • switch argument {
      • case "--help", "-h":
        • help()
      • case "--bgcolor":
        • let bgcolor = popString(warning:"Background color must be color values.")
        • backgroundColor = stringToColor(color:bgcolor)
      • case "--border":
        • borderWidth = 0.5/12
      • case "--fgcolor":
        • let fgcolor = popString(warning:"Background color must be color values.")
        • foregroundColor = stringToColor(color:fgcolor)
        • fontAttributes[NSAttributedString.Key.foregroundColor] = foregroundColor
      • case "--font", "-f":
        • let fontName = popString(warning:"Font requires a font name.")
        • //is there a font size?
        • fontSize = popFloat(defaultValue: fontSize)
        • guard let font = NSFont(name: fontName, size: fontSize) else {
          • print("Unknown font " + fontName)
          • exit(0)
        • }
        • fontAttributes[NSAttributedString.Key.font] = font
      • case "--restrict", "-r":
        • restrictedWidth = popFloat(warning:"--restrictwidth requires a value")
      • case "--wrap", "-w":
        • maximumWidth = popFloat(defaultValue:-1)
        • wrapOptions: while arguments.count > 0  {
          • switch arguments[0] {
            • case "center":
              • alignment = NSTextAlignment.center
            • case "double":
              • doublespace = true
            • case "hyphenate":
              • hyphenate = true
            • case "indent":
              • indent = true
            • case "justify":
              • alignment = NSTextAlignment.justified
            • case "right":
              • alignment = NSTextAlignment.right
            • default:
              • break wrapOptions
          • }
          • arguments.removeFirst()
        • }
      • default:
        • if imageFile == "" {
          • imageFile = argument
        • } else {
          • help(message:"Unknown option: " + argument)
        • }
    • }
  • }
  • borderWidth *= fontSize
  • var borderPadding = borderWidth * 8
  • //validate filename
  • let validFiletypes = [".jpg", ".tiff", ".png"]
  • if imageFile == "" {
    • help(message:"An image file is required.")
  • } else if !validFiletypes.contains(where:imageFile.hasSuffix) {
    • help(message:"Images must be png, jpg, or tiff")
  • }
  • //read lines from standard input
  • var lines = [String]()
  • var columnWidths = [CGFloat]()
  • var tabAlignments = [NSTextAlignment]()
  • var isTable = false
  • while let line = readLine() {
    • lines.append(line)
    • //calculate column widths for potential tab stops
    • if line.contains("\t") {
      • //It's only a table if there are tabs inside at least one line
      • if line.trimmingCharacters(in:CharacterSet(charactersIn: "\t")).contains("\t") {
        • isTable = true
      • }
      • let columns = line.components(separatedBy: "\t")
      • for (index, column) in columns.enumerated() {
        • let styledColumn = NSAttributedString(string:column, attributes:fontAttributes)
        • let newColumnWidth = styledColumn.size().width
        • if columnWidths.count > index {
          • let existingColumnWidth = columnWidths[index]
          • if newColumnWidth > existingColumnWidth {
            • columnWidths[index] = newColumnWidth
          • }
        • } else {
          • columnWidths.append(newColumnWidth)
          • //columns other than the first are aligned right by default
          • tabAlignments.append(NSTextAlignment.right)
        • }
        • //if column contains anything other than number-related characters, align it to the default
        • if column.range(of: "^[0-9,$.-]*$", options: .regularExpression) == nil {
          • tabAlignments[index] = NSTextAlignment.left
        • }
      • }
    • }
  • }
  • //set default width from font size
  • if !isTable && maximumWidth < 0 {
    • maximumWidth = NSAttributedString(string:"b", attributes:fontAttributes).size().width*50
  • }
  • //set up tab stops if necessary
  • if isTable {
    • let paragraphStyle = NSMutableParagraphStyle()
    • paragraphStyle.tabStops = []
    • var tabStop:CGFloat = 0.0
    • for index in 0...columnWidths.count-2 {
      • tabStop += columnWidths[index] + fontSize
      • //tab at index is for data at index+1 so add width if right aligned
      • let tabAlignment = tabAlignments[index+1]
      • if tabAlignment == NSTextAlignment.right {
        • tabStop += columnWidths[index+1]
        • columnWidths[index+1] = 0
      • }
      • paragraphStyle.tabStops.append(NSTextTab(textAlignment: tabAlignment, location: tabStop, options: [:]))
    • }
    • paragraphStyle.headIndent = columnWidths[0]
    • fontAttributes[NSAttributedString.Key.paragraphStyle] = paragraphStyle
  • } else if maximumWidth > 0 {
    • let paragraphStyle = NSMutableParagraphStyle()
    • paragraphStyle.alignment = alignment
    • if indent == true { paragraphStyle.firstLineHeadIndent = fontSize }
    • if hyphenate == true { paragraphStyle.hyphenationFactor = 1.0 }
    • if doublespace == true { paragraphStyle.lineSpacing = fontSize }
    • fontAttributes[NSAttributedString.Key.paragraphStyle] = paragraphStyle
  • }
  • //create text for image
  • var imageLines = NSMutableAttributedString()
  • for (index, text) in lines.enumerated() {
    • //if this is indented, do not include blank lines
    • if text == "" && indent {
      • continue
    • }
    • imageLines.append(NSAttributedString(string:text, attributes:fontAttributes))
    • if index < lines.endIndex-1 {
      • imageLines.append(NSAttributedString(string:"\n", attributes:fontAttributes))
    • }
  • }
  • //create the image and rect
  • var outputSize = imageLines.size()
  • if maximumWidth > 0 && outputSize.width > maximumWidth-borderPadding*2 {
    • outputSize.width = maximumWidth-borderPadding*2
    • outputSize.height = 0.0
    • let neededRect = imageLines.boundingRect(with:outputSize, options:NSString.DrawingOptions.usesLineFragmentOrigin)
    • outputSize.width = neededRect.width+borderPadding*2
    • outputSize.height = neededRect.height+borderPadding*2
  • } else if borderPadding > 0 {
    • outputSize.width += borderPadding*2
    • outputSize.height += borderPadding*2
  • }
  • var textRect = NSRect(x: borderPadding, y: borderPadding, width: outputSize.width-borderPadding*2, height: outputSize.height-borderPadding*2)
  • //draw the text in the image/rect
  • var outputImage = NSImage(size:outputSize, flipped: false) { (outputRect) -> Bool in
    • //fill the image with the background color
    • backgroundColor.setFill()
    • outputRect.fill()
    • //draw the text
    • imageLines.draw(in: textRect)
    • //draw the border if requested
    • if borderPadding > 0 {
      • foregroundColor.setFill()
      • outputRect.frame(withWidth:borderWidth)
    • }
    • return true
  • }
  • if restrictedWidth > 0 && outputImage.size.width > restrictedWidth {
    • outputImage = scaleImage(image:outputImage, width:restrictedWidth)
  • }
  • //convert image to desired format
  • var imageData = outputImage.tiffRepresentation
  • if !imageFile.hasSuffix(".tiff") {
    • let bitmapVersion = NSBitmapImageRep(data: imageData!)
    • var filetype = NSBitmapImageRep.FileType.png
    • if imageFile.hasSuffix(".jpg") {
      • filetype = NSBitmapImageRep.FileType.jpeg
    • }
    • imageData = bitmapVersion!.representation(using: filetype, properties:[:])
  • }
  • //write image to file
  • do {
    • try imageData!.write(to: NSURL.fileURL(withPath: imageFile))
  • } catch {
    • print("Unable to save file: ", imageFile)
  • }

To be honest, as much fun as it was to write it and play around with it, I’m not sure how often I’ll use this script for anything other than what I originally wrote it for, converting tables to text for Smashwords. Most of the time if I have text I want converted to an image, the text is sitting on the screen ready for a screenshot. And it’s very rare I want to convert text to image anyway; text is almost always far more useful than an image of text is.

But it is a fun alternative to the aligntabs and makeHTMLTable scripts from 42 Astoundingly Useful Scripts and Automations for the Macintosh. It may turn out to be useful for text messages, which don’t work well with either text tables or html tables.

  1. The reason there is no kindle version of the ebook for 42 Astounding Scripts is that I was unable to find a way to allow kindle readers to copy code without it getting mangled.

    ↑

  2. Probably by running the text through aligntabs from 42 Astounding Scripts, but since the tests showed convert wasn’t a good solution, I never got that far.

    ↑

  3. Or if started without piping, what you type until you type CTRL-D.

    ↑

  4. The first column is always aligned left, sort of. The first column is the one that comes before any tabbed text. It would be possible to detect if the first column ought to be aligned right and then add a tab to the beginning of each line, but I haven’t had a need for that yet.

    Despite what it might look like in this script, I do prefer to wait until I have a need for a feature before I add the feature, because otherwise I’m likely to miss something important about the feature and program it incorrectly.

    ↑

  5. Technically, the width of 50 letter ‘b’s. I tried several characters to get an average, and the letter ‘b’ seemed just right. I mostly tested in Times New Roman and Adobe Garamond, so your mileage may vary.

    ↑

  6. Nisus Writer Pro has Perl built-in, but any attempt to run a command-line program using, for example, Perl’s backtick operator will fail due to the sandbox.

    ↑
  1. <- Tandy CoCo Tools
  2. Avoiding lockFocus ->