5 things to know about the GitHub Asciidoctor upgrade

by Dan Allen -

After much anticipation, GitHub recently upgraded Asciidoctor to the 0.1.4 release, thanks to @bkeepers!

Even bigger news than the upgrade, syntax highlighting is enabled in source listings! If you need to see it to believe it, I present you proof.

syntax highlight sample github
Figure 1. Syntax highlighting in an AsciiDoc source block on GitHub

Since much has changed in Asciidoctor since version 0.1.0—​the previous version deployed—​we’ll look at 5 things to know about this upgrade to make the most of AsciiDoc rendering on GitHub.

1. Source code syntax highlighting

First and foremost, syntax highlighting of source code is finally enabled! The splashes of color in the source listings are added by the Pygments syntax highlighter.

You don’t have to do anything special to start using this feature. Just add a source block to your AsciiDoc file using:

  1. the listing block syntax

    [source,ruby]
    ----
    puts ['AsciiDoc', 'GitHub', 'AsciiDoc on GitHub'].map {|item|
      "I use #{item}!"
    } * "\n"
    ----
  2. the open block syntax

    [source,ruby]
    --
    puts ['AsciiDoc', 'GitHub', 'AsciiDoc on GitHub'].map {|item|
      "I use #{item}!"
    } * "\n"
    --
  3. or the fenced code block syntax (ala GitHub-flavored Markdown)

    ```ruby
    puts ['AsciiDoc', 'GitHub', 'AsciiDoc on GitHub'].map {|item|
      "I use #{item}!"
    } * "\n"
    ```

Here’s how it looks when rendered:

puts ['AsciiDoc', 'GitHub', 'AsciiDoc on GitHub'].map {|item|
  "I use #{item}!"
} * "\n"

You can find the list of languages Pygments recognizes on the lexers page in the Pygments documentation.

WHOA!!!!! OMG!!! WOOOOOOOOOOOO AAAAAAAAAAAAA WOW OOOOOO YAAAAAAAAAAA!!!! DOUBLE RAINBOW!!!

2. Document title and footnotes

You’ll be glad to know that GitHub now shows the document title (level 0 section title) as a <h1> heading, if your AsciiDoc file has one. Previously, the document title was ignored, requiring you to use a workaround if you wanted your rendered document to start with a heading (as most people do).

What’s more, GitHub now renders footnotes where you might expect to find them, at the bottom of the document[1]. Unfortunately, the link from the reference to the footnote does not work because the HTML sanitizer wipes out the id attributes on elements (which serve as the anchors). But hey, at least you know to look for the footnote. We’ll get those links fixed in the next upgrade.

I would like to be able to tell you that include files now work, but that news will have to wait for another day once we’ve alleviated the security concerns. Until then, we at least have a solution that improves the reading experience.

If you use the include directive in your AsciiDoc, Asciidoctor replaces the directive with a link to the target file, making it easy to navigate between files from the GitHub interface. You can see an example in this sample AsciiDoc file.

4. Compliant AsciiDoc topped with syntactic sugar

The upgrade to Asciidoctor 0.1.4 brings all sorts of improvements to the AsciiDoc syntax. AsciiDoc documents are now parsed as accurately, if not more so, than if AsciiDoc Python were used. That means that AsciiDoc on GitHub is the real deal for the first time since AsciiDoc Python was put out to pasture.

Improved compliance

Here are a couple of areas of the syntax that now get parsed correctly:

  • attribute entries take effect at the location they are found in the document

    :speaker: John
    :subject: Sarah
    
    ``Hi {subject}!'', shouted {speaker}.
    
    :speaker: Sarah
    :subject: John
    
    ``Hey there, {subject}!'', replied {speaker}.
  • attribute entries inside verbatim content are now properly ignored

    [literal]
    :name: not an attribute entry
  • explicit substitutions can be applied to any block using the subs attribute

    [source,xml,subs="attributes,verbatim"]
    --
    <dependencies>
      <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-context</artifactId>
        <version>{spring-version}</version>
        <scope>runtime</scope>
      </dependency>
    </dependencies>
    --
  • an alternate context may be assigned to block-level content using the first positional attribute, as permitted by the AsciiDoc syntax

    For example, an open block can masquerade as a source block, as shown here:

    [source,ruby]
    --
    puts 'Hello, GitHub!'
    --
  • lists are parsed more accurately, especially those containing complex content

Sugar on top

On top of this compliant behavior is a sweet layer of Asciidoctor goodness. Such enhancements include:

  • shorthand id, role and options syntax for blocks

    [sidebar#audience.text-justify]
    --
    This content appears in a sidebar with id "audience" and justified text.
    --
  • parsing of select Markdown content, including headings and quote blocks

    ## Asciidoctor speaks Markdown
    
    > Did you here that Asciidoctor understands how to parse basic Markdown?
    
    You bet I did!
  • implicit table header row and implicit content delimiter

    ,===
    Project, Language
    
    Asciidoctor, Ruby
    AsciidoctorJ, Java
    ,===
  • no dropped lines for missing attributes

    Although {foobar} cannot be resolved, this line is rendered as is.
  • multiple authors in the document header, separated by semi-colons

    = Document Title
    First Author; Second Author; Third Author
  • you can drop the quotes around values in block attributes if the value does not contain a comma or space

    [cols=3,width=50%]
  • custom macros, including menu, btn, kbd and icon

    :experimental:
    
    Open a new tab by pressing kbd:[Ctrl+T] or selecting menu:File[New].

We could go on, but I’ll stop there and let you explore the many additional layers of sweetness that Asciidoctor adds.

Although the AsciiDoc syntax is correctly parsed and rendered, the output doesn’t always look as you might expect. The challenge we face is that the HTML output from Asciidoctor often relies on the stylesheet to complete the UI pattern being represented. However, those styles are not present on GitHub.

What makes things even more difficult is that the content is passed through an HTML sanitizer that strips away most of the HTML attributes, so even if we could map to existing CSS classes in the GitHub, we can’t because the class attribute gets stripped. In a future upgrade, we’ll work on finding a way to make these UI patterns work, such as admonition blocks.

5. Table of contents

Last, but not least, you can now include a table of contents in the output to help navigate those larger documents.

The first thing you need to do to enable the table of contents on GitHub is set the toc attribute in the document header.

= Document Title
:toc:

Next, you need to set the toc-placement attribute to preamble or unset the attribute.

= Document Title
:toc:
:toc-placement: preamble

If you choose preamble, the table of contents will be rendered under the preamble but before the first section. If you unset the attribute instead, you will need to place the table of contents in the document manually using the toc::[] block macro.

= Document Title
:toc:
:toc-placement!:

toc::[]

You’ll notice that superfluous bullets appear in the table of contents when rendered on GitHub. This behavior can be explained by the fact that the necessary styles are missing, as described at the end of the last section.

Long live AsciiDoc on GitHub!

The recent Asciidoctor upgrade on GitHub certainly brought with it a lot of improvements. You can see more examples showcasing what’s available in the asciidoc-samples repository.

If you already have AsciiDoc content on GitHub and you aren’t seeing the new features, you probably need to make a modification to the file to boot the rendered file from the cache and force GitHub to regenerate it.

If something still doesn’t render to your satisfaction, you have one more “escape route” option. There are three implicit attributes you can consult to conditionally include or exclude content shown in the table below. The first two are only assigned in the GitHub environment.

Attribute name Attribute value

env

github

env-github

empty string

asciidoctor-version

0.1.4

If you want to render different content when on GitHub, simply use the ifdef and ifndef preprocessor directives:

ifdef::env-github[]
I'm on GitHub!
endif::[]
ifndef::env-github[]
I'm clearly somewhere else right now.
endif::[]

If you only want to render content when the version of Asciidoctor is upgraded, then use the ifeval preprocessor directive:

ifeval::["{asciidoctor-version}" > "0.1.4"]
Asciidoctor has been upgraded again!
GitHub is now running Asciidoctor {asciidoctor-version}.
endif::[]

That wraps up our brief overview of the new AsciiDoc experience on GitHub. AsciiDoc on GitHub is no second class citizen to Markdown anymore. It’s the real deal, syntax highlighting and all.

I’d like to thank everyone who helped make this upgrade happen, including Ryan Waldron, Matthew McCullough, Tim Berglund, Garen Torikian, GitHub Security and the man who it, Brandon Keepers. Of course, I’d also like to thank everyone in the Asciidoctor community who advocates for AsciiDoc and helps make Asciidoctor awesome.

Happy documenting!


1. I’m a little footnote, short and stout.

Easier Documentation, Simpler Websites And Faster Collaboration @ OSCON 2013

by Dan Allen -

Do you find producing and collaborating on documentation or web content difficult? At OSCON 2013, we’ll work with you to reduce the friction of creating content by simplifying your content workflow, teaching you how to collaborate more efficiently and helping to make writing and publishing enjoyable!

OSCON banner

Join us on Tuesday, July 23 at our workshop titled Easier Documentation, Simpler Websites And Faster Collaboration to learn about tools and techniques that lighten your documentation workload and jumpstart your content strategy. To attend, register for OSCON 2013, then sign up for the workshop!

TIP: Use code ARQIKE for a 20% discount off registration!

How?

It begins with writing content…​

…​in the same format you use to write e-mail.

Plain text *embellished* with _subtle_ markup.

Indeed, that’s the focus of the lightweight markup language AsciiDoc. The AsciiDoc syntax feels natural and that keeps you focused on the content. Content written in AsciiDoc is easy to read and edit in raw form, which facilitates collaboration. It can also be translated into HTML for presentation, hello web publishing!

Zurb Foundation Yeti

Next, we do some baking.

After composing some documentation in AsciiDoc, we’ll pull together our content into a cohesive website using the static site generators Awestruct and Jekyll. Then we’ll leverage CSS frameworks like Bootstrap and Zurb Foundation to dress up the content with an elegant look and feel.

It’s push to publish!

We’ll then publish our website to GitHub Pages using a single command, git push.

That’s when the real fun starts. From GitHub, we can collaborate on the content using git and the web-based collaboration tools in the GitHub interface.

Revision history in GitHub
The revision history is safely tucked away in the git repository.

Don’t forget to chunk!

There are billions of devices connected to the web. And more are coming. Regardless of how the device market changes over time, the challenge for us remains the same. We need to get our content on all of them.

AsciiDoc is more than just a lightweight markup language. It enables and encourages us to write reusable, structured content.

Chunking Reusable content can be mixed and matched with other content to create a composition that suits a device, context or situation. It’s also a way to keep our content DRY (Don’t Repeat Yourself) so we don’t have to update the same content in a dozen places, or even two.

Structured content is meaningful, semantic content that is flagged using metadata. We can do lots of things with a document if we know what’s in it. We can pull out a lead or excerpt. We can hide content when it isn’t needed. We can present a structure and allow the user to drill into what they want to see.

Content no longer lives on a “page”. The future unit of publishing is the “chunk”. These smaller, discrete sources can be tuned to specific platforms or aggregated using automation.

The web is changing. It’s left the desktop. It’s now a road warrior. The way we think about content publishing must adapt. More than ever before, we must separate content from presentation.

Recap

Writing documentation is hard. So we don’t do it often enough. And yet, we’re skipping out on the best opportunity to reach and influence our target audience. Let’s do something about it.

  • lightweight markup makes writing documentation easier

  • static site generators and CSS frameworks make websites simpler

  • git and GitHub make collaborating on content and publishing it online faster

…​all powered by open source software:

Ruby

Git

AsciiDoc / Asciidoctor

Awestruct

Jekyll

Haml / Slim

SASS / Compass

Zurb Foundation

Travis CI

Let’s bake better documentation, together. Documentation that’s reusable and structured.

What, Where and When?

Title:

Easier Documentation, Simpler Websites And Faster Collaboration

Date:

Tuesday, July 23, 2013

Time:

1:30 - 5:00 PM (3h 30m)

Room:

D139/140, Oregon Convention Center

Type:

Workshop

Category:

Tools & Techniques

You can find all the details about the workshop, including the session abstract, on the official session page.

Who?

Attendees

Open source community members like you who are passionate about documentation and web publishing.

Not sure if that’s you? Do you contribute to, maintain or organize any of the following?

  • manuals, user guides, tutorials or READMEs

  • news, press releases or announcements

  • articles or books

  • brochures or press kits

  • conference or event information

  • request for proposals (RFPs)

  • resume or personal site

If you nodded, then you’re one of us :)

Trainers

Dan Allen Dan Allen

Dan is an open source advocate, community catalyst, author and speaker. He proudly pursues these passions as a Red Hat employee and community member.

In his role as Principal Software Engineer at Red Hat, he leads the Asciidoctor project and serves as the community manager for Arquillian. He draws on these experiences to help make a variety of open source projects wildly successful. Besides drinking a Trappist beer or indulging in Belgian chocolate, there’s nothing he’d rather do.

Sarah White Sarah White

Sarah is the content strategist for both the Arquillian and Asciidoctor projects—​an ideal position for someone passionate about open source, alien invasions and writing.

If there’s room for improvement, Sarah will find it. Lots of it.

Long ago, in a not-too-distant galaxy, she assessed hazardous waste sites and tracked pesticide routes through watersheds. So she knows a thing or two about identifying and eradicating stuff that kills, including software bugs and poor documentation.

Prerequisites

Knowledge of HTML and being comfortable using the commandline are both essential. Some knowledge of git and Ruby is useful, though a novice should be able to pick up the necessary training “on the job”.

Reading list

We’ve prepared some reading material that will give you a better idea of what the session is about and what you’ll be learning. These resources should also help you continue to learn about the subject after the workshop is over.

A full list of resources is available on the Reference Resources page on the workshop’s wiki.

We look forward to seeing you at OSCON!


AsciiDoc, powered by Asciidoctor, returns to GitHub and its 5+ million repositories

by Dan Allen -

Since early December and throughout the holidays, GitHub’s Ryan Waldron, Red Hat’s Dan Allen and other contributors have collaborated on Asciidoctor (project site, code repository), a new open source implementation of AsciiDoc written purely in Ruby[1].

The implementation kicked off with two core goals in mind:

  1. Improve the rendering of AsciiDoc source files in git repositories and gists on GitHub

  2. Bring AsciiDoc to the Ruby world by offering the first compliant Ruby port of the markup language

After a lot of hard work and a thorough security audit, we’re thrilled to announce that…​

Note Asciidoctor is now deployed to render AsciiDoc markup in any of the 5 million+ repositories on GitHub.

Asciidoctor 0.1.0 is also available on rubygems.org and can be installed locally via RubyGems (gem install asciidoctor).

About AsciiDoc

AsciiDoc is a lightweight markup language akin to Markdown. Going beyond Markdown, AsciiDoc supports all the structural elements necessary for drafting articles, technical manuals and books. Several O’Reilly authors, including Matthew McCullough, Tim Berglund and Matt Neuburg, have used it to write books for that iconic technical library. AsciiDoc is fully capable of serving as a shorthand replacement for DocBook, which it can also output.

DocBook is nice, but (like XML) it is not meant for editing nor for merging changes (by humans). Using AsciiDoc (which translates to DocBook perfectly) is a much easier way of developing.

— Dag Wieers

One way to think of the relationship between AsciiDoc and DocBook is that AsciiDoc is to DocBook as RelaxNG Compact is to XML Schema. With AsciiDoc, you drop the angled-brackets (i.e., XML), but not the semantics.

Here’s an example of a document written in AsciiDoc, shown above the output it produces:

(You don’t want to see how the DocBook looks for this example. Let’s just say, there isn’t enough room on the screen to show it to you.)

Source document

= Asciidoctor
Ryan Waldron, Dan Allen

http://asciidoctor.org[Asciidoctor] is a native Ruby processor
for converting AsciiDoc source files and strings into HTML 5,
DocBook 4.5 and other formats.

== Installation

You can install the Asciidoctor RubyGem using the +gem+ command:

 gem install asciidoctor

On Fedora, you can install via an RPM package
(https://bugzilla.redhat.com/show_bug.cgi?id=889011[pending])
using the +yum+ command:

 yum install rubygem-asciidoctor

Now you are ready to start using Asciidoctor!

== Benefits

Asciidoctor is:

* compliant
* fast
* secure
* stable

'Give it a try!'

Rendered HTML Output

Asciidoctor

Ryan Waldron, Dan Allen

Asciidoctor is a native Ruby processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats.

Installation

You can install the Asciidoctor RubyGem using the gem command:

gem install asciidoctor

On Fedora, you can install via an RPM package (pending) using the yum command:

yum install rubygem-asciidoctor

Now you are ready to start using Asciidoctor!

Benefits

Asciidoctor is:

  • compliant

  • fast

  • secure

  • stable

Give it a try!

The AsciiDoc project, written in Python, was the only implementation of the AsciiDoc syntax…​until the arrival of "the doctor".

The Asciidoctor implementation

Asciidoctor is the first project to implement the AsciiDoc syntax in another language, in this case Ruby. It’s an open source library (MIT licensed) published as a RubyGem to rubygems.org. The project recently moved to the Asciidoctor organization on GitHub to sustain its rapid growth.

While Asciidoctor aims to offer full compliance with the AsciiDoc syntax (with some exceptions, noted in the project README), it’s more than just a clone.

Built-in and custom templates

Asciidoctor uses a set of built-in ERB templates to generate HTML 5 and DocBook 4.5 output that is structurally equivalent to what AsciiDoc produces. Any of these templates can be replaced by a custom template written in any template language available in the Ruby ecosystem. Custom template rendering is handled by the Tilt template abstraction library, a project created by another GitHubber, Ryan Tomayko.

Parser and object model

Leveraging the Ruby stack isn’t the only benefit of Asciidoctor. Unlike the AsciiDoc Python implementation, Asciidoctor parses and renders the source document in discrete steps. This makes rendering the document optional and gives Ruby programs the opportunity to extract, add or replace information in the document by interacting with the document object model Asciidoctor assembles. Developers can use the full power of the Ruby programming language to play with the content in the document.

Performance and security

No coverage of Asciidoctor would be complete without mention of its speed. Despite not being an original goal of the project, Asciidoctor has proven startlingly fast. It loads, parses and renders documents at least 25 times as fast as the Python implementation. That’s good news for developer productivity and good news for GitHub or any server-side application that needs to render AsciiDoc markup. Asciidoctor also offers several levels of security, further justifying its suitability for server-side deployments.

Beyond Ruby

Asciidoctor’s usage is not limited to the Ruby community. Thanks to JRuby, a port of Ruby to the JVM, Asciidoctor can be used inside Java applications as well (and eventually in Java build tools like Apache Maven). Asciidoctor also ships with a command-line interface (cli), written by Red Hat’s Jason Porter. The Asciidoctor cli, asciidoctor, is a drop-in replacement for the asciidoc.py script from the AsciiDoc Python distribution.

The future of Asciidoctor and AsciiDoc

The future is bright for AsciiDoc. Despite being a seasoned, 10-year-old markup language, adoption of AsciiDoc has never been stronger. The developers have lots of ideas about how to improve and extend Asciidoctor, some of which push beyond the AsciiDoc syntax.

  • If you’re interested in using AsciiDoc, head over to GitHub and create a new file in one of your repositories or gists using the file extension .asciidoc or .adoc.

  • If you’re interested in contributing to Asciidoctor, in turn helping to move AsciiDoc forward, your participation and feedback is welcome!

Write docs with pleasure!

This article was composed in AsciiDoc and rendered using Asciidoctor.

1. The Asciidoctor code base emerged from a prototype that GitHub developers created last year to produce the git man pages shown on the git website.

  • 1 of 1