Reformatting old text books in vim

I’ve found a few old books from the 1990s, which are in the simple txt format. Since there is no explicit formatting possible, my PocketBook 740 Color e-reader formats the files not in the best way. Hard-wrapped paragraphs separated by empty lines are left-aligned and aren’t reflown in the reader, so a lot of space is wasted on the right. I couldn’t quickly find a program that would reformat such files; Calibre’s ebook converter didn’t do what I needed. In the end, I was able to cleanup such books in vim with a few commands.

Source

Here’s an example of the source text that I have:

Title

Author

CHAPTER 1

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

-- Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore
eu fugiat nulla pariatur.

* * *

-- Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia
deserunt mollit anim id est laborum.

Sed ut perspiciatis, unde omnis iste natus error sit voluptatem accusantium
doloremque laudantium, totam rem aperiam eaque ipsa, quae ab illo inventore
veritatis et quasi architecto beatae vitae dicta sunt, explicabo.

CHAPTER 2

Nemo enim ipsam voluptatem, quia voluptas sit, aspernatur aut odit aut fugit,
sed quia consequuntur magni dolores eos, qui ratione voluptatem sequi nesciunt,
neque porro quisquam est, qui dolorem ipsum, quia dolor sit amet consectetur
adipisci[ng] velit, sed quia non numquam [do] eius modi tempora inci[di]dunt, ut
labore et dolore magnam aliquam quaerat voluptatem.

* * *

--- Temporibus autem quibusdam et aut officiis debitis aut rerum necessitatibus
saepe eveniet, ut et voluptates repudiandae sint et molestiae non recusandae.

Itaque earum rerum hic tenetur a sapiente delectus, ut aut reiciendis
voluptatibus maiores alias consequatur aut perferendis doloribus asperiores
repellat.

The goal is to unwrap paragraphs into long lines, remove empty lines between paragraphs within each section, squash multiple empty lines into one between each section and squash multiple dialog dashes into one.

Commands

Here are the commands to reformat such a file:

:setl tw=99999
:4,$g/\v^(.+)\n\ze(.+)/normal gqap
:%s/\v^-{2,} /- /
:4,$s/\v^$\n(.)/\1/
:%s/\v^$(\n^$)+//

Explanation:

:setl tw=99999 sets the textwidth property to a big value so that the vim’s reformatter will join hard-wrapped lines within a paragraph;
:4,$g/\v^(.+)\n\ze(.+)/normal gqap does the reformatting of paragraphs so that they are in one line each;
- 4,$ is the range of the command, to ignore the book title;
- the starting \v in the regexps is to make the special characters work like in regexps without extra backslashes;
- ^(.+)\n\ze(.+) looks for two adjacent non-empty lines (without an empty line in-between);
- normal gqap formats the current paragraph for every matching paragraph.
:%s/\v^-{2,} /- / squashes multiple (dialogue) dashes at the beginning of a line into one;
:4,$s/\v^$\n(.)/\1/ removes a signle empty line before a paragraph by replacing an empty line ^$\n with whatever follows it;
:%s/\v^$(\n^$)+// squashes multiple adjacent empty lines into one.

I learned about the :global command from the very interesting tutorial Vim Waz ‘Ere.

Result

The result is:

Title

Author

CHAPTER 1

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
- Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.

* * *

- Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Sed ut perspiciatis, unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam eaque ipsa, quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt, explicabo.

CHAPTER 2

Nemo enim ipsam voluptatem, quia voluptas sit, aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos, qui ratione voluptatem sequi nesciunt, neque porro quisquam est, qui dolorem ipsum, quia dolor sit amet consectetur adipisci[ng] velit, sed quia non numquam [do] eius modi tempora inci[di]dunt, ut labore et dolore magnam aliquam quaerat voluptatem.

* * *

- Temporibus autem quibusdam et aut officiis debitis aut rerum necessitatibus saepe eveniet, ut et voluptates repudiandae sint et molestiae non recusandae.
Itaque earum rerum hic tenetur a sapiente delectus, ut aut reiciendis voluptatibus maiores alias consequatur aut perferendis doloribus asperiores repellat.

The text is now properly displayed on the reader.

Improvements

I’ve tried making it one command:

:setl tw=99999 | 4,$g/\v^(.+)\n\ze(.+)/exe "normal gqap" | %s/\v^-{2,} /- / | 4,$s/\v^$\n(.)/\1/ | %s/\v^$(\n^$)+//

, but a lot more lines are joined together before the last step. However two commands work:

:setl tw=99999 | 4,$g/\v^(.+)\n\ze(.+)/exe "normal gqap" | %s/\v^-{2,} /- /
:4,$s/\v^$\n(.)/\1/ | %s/\v^$(\n^$)+//

Finally, it’s possible to automate this processing even more by asking vim to edit a file and save the result:

$ vim -u NONE -c 'setl tw=99999 | 4,$g/\v^(.+)\n\ze(.+)/exe "normal gqap"' -c '%s/\v^-{2,} /- / | 4,$s/\v^$\n(.)/\1/ | %s/\v^$(\n^$)+//' -c 'execute "saveas " .. expand("%") .. "_" | quit' test.txt

Thus if you have the input file test.txt, you’ll get the fixed output file test.txt_. Success!

KISS 🇺🇦

Stop the war!

Reformatting old text books in vim

Source

Commands

Result

Improvements

Comments