Emacs Allout Outliner

Allout is an outlining extension for the Emacs programming editor which I created and continue to develop. Allout has been included as part of the official Emacs distribution since 1993.Contents

Allout is an extensive outline editing mode module that I developed and maintain for the Emacs programming-editor. Allout is distributed with Emacs. As of Emacs 24, a companion visual enhancement which I developed, allout-widgets, is also included with the Emacs distribution.

I first publicly released allout in 1991 (via Usenet). It has been included with the official GNU Emacs distribution since 1993.  In 2008 I got direct committer access to maintain allout in the GNU repository. As of Feb, 2011, I added allout-widgets to the Emacs source code trunk, bringing the distribution up-to-date with the version of allout that I use daily. Hence the GNU repository is the place to go for the latest allout version - see the next topic, Getting The Latest Versions, and the emacs wiki Emacs Development Sources page for general info about using the repository. See further below for package background and activation info.

Getting the Latest Allout Versions

Recent Emacs versions are distributed with recent allout versions. If for some reason you want to get a different allout version than that which comes with your Emacs instance you can get allout.el and allout-widgets.el (plus incidentals) from the emacs source code repository trunk. See Activation, below, for activation instructions. I don't guarantee that all of this info is up to date, but the repository versions at least should be.

What's Emacs?

Emacs is a supremely extensible open-source text editor used principally by programmers (XKCD) . It is essentially the first free-software creation of Richard Stallman, who started and has been a primary driving force in the free / open-source software movement. Being open-source and so extensible, it has always been the product of collaboration, with numerous people contributing to its ongoing development. Due to some fundamental design choices, like user control through keyboard chords, it's subject to user favor or objection that approach religion-level fervor. Over the years I've developed a few extensions which are delivered with the editor, including particularly Allout mode for editing and navigating outlines. I continue to use Emacs and Allout extensively.

What's Allout?

Allout is an emacs editor mode useful for navigating and editing hierarchically organized text outlines. I based it on a more rudimentary outline mode that also comes with Emacs. They both use configurable conventions for delimiting topic items, and so can be used with text that has its own structuring constaints, like programming languages. Allout operates as a minor mode so it can be used in combination with other emacs modes.

By configuring the topic items to start with a particular programming language's line comments, allout can be used in combination with programming language modes to organize programming code as outlines. The allout (and allout-widgets) emacs lisp code are, themselves, examples of such structuring.

Allout provides structured outline editing beyond the original outline mode, including coherent promotion and demotion of topics that affects contained topics accordingly, coherent cut and paste of topics (and offspring, accordingly) across outline depths, automatic item numbering, and much more.

Allout also provides for Gnu Privacy Guard (GnuPG, GPG) topic encryption, enabling numerous distinct (symmetric and/or key-pair) encrypted entries within a single file. It's very handy for securely keeping track of passwords, accounts, and other sensitive information within outline-organized text files, depending on the underlying GPG tools for security.

See below for a more details (but not complete) list of distinct Allout features.

Features

  • Easy topic-oriented navigation, exposure adjustment, and editing of text files formatted to distinguish topic elements according to customizable criteria
    • Editing and exposure adjustment applies coherently over the nested items, eg promotion and demotion of a topic tree as a whole, and likewise copy or cut and paste of topics
  • Configurable per-file initial exposure settings, so the structure is presented as you wish on initially visiting a file
  • Incremental search and query-replace with dynamic exposure of hidden targets
  • Easy assignment of distinctive bullet characters to special topics, for distinguishing them, including...
    • ... assignment of a numbering bullet characters to siblings, for automatic topic-numbering maintenance
    • ... and assignment of programming-language comment characters as topic prefixes, so allout can be used to structure and interact with programming and other code as outlines
  • "Hot-spot" operation, for single-keystroke outline-oriented operation - navigation, exposure control, and editing - when the cursor is situated over a topic's allout prefix
  • Topic-specific GnuPG encryption, with option for both symmetric and key-pair protocols, enabling for easy and secure encryption and decryption of multiple distinct items within an outline file
  • Easy rendering of exposed outline sections into simple textual outline styles, including numbered outlines, latex formatting, white-space indented, etc.
  • Careful attention to whitespace -- enabling blank lines between items and maintenance of hanging indentation (in paragraph auto-fill and across topic promotion and demotion) of topic bodies consistent with indentation of their topic header.

The addition of allout-widgets adds tree-branching guide lines and iconic bullet characters, making outline structure more clear and easy to recognize.

Activation

You can configure (the recent versions of) allout and allout-widgets to auto-activate when visiting recognizably allout-structured files by using emacs' user customizations. Do:

Esc-x customize-group <ret> allout

and look for Allout Auto Activation, and similarly for Allout Widgets Auto Activation in the allout-widgets subgroup.

There is an Emacs Wiki page on allout, including some background info, links to this page, contributed guidance, and so forth.

User feedback and questions

A remark from one enthusiastic allout user:

Using allout is like putting on glasses - I've been able to improve and clarify code in every file in which I've used it. Sure, any outline mode could help with that, but they all have arcane or busy navigation & show/hide commands. The genius of allout's hot-spot navigation makes it all as easy as putting on a pair of glasses. Hmm... that sounds like promotional copy - feel free to use it if it answers a need :-) Bill W.

From: Stefan, Date: Sun, 18 Jun 2006 16:10:37 -0400
Subject: automatic loading of auto-fill mode

I am trying out the latest/greates (!) allout-mode version that I have downloaded from this site. I try to adapt it to my needs. When I work with text files I prefer explicit filling of paragraphs and therefore do not use the auto-fill-mode. When I load allout-mode auto-fill-mode gets loaded automatically. Is auto-fill mode essential for allout-mode to work. If not is it possible to make the automatic loading of auto-fill mode configurable - or are there any other reasons why this would not be a good idea. Thanks for you answer and many thanks for allout-mode - Stefan

From: klm, Date: Sun, 18 Jun 2006 18:12:15 -0400
Subject: automatic loading of auto-fill mode

i've uploaded a new allout version which includes, among other things, a new custom configuration variable, `allout-inhibit-auto-fill'. setting this will turn off auto-fill in the buffer where you set it (if you use 'Esc-x set-variable' or 'Esc-: (setq ...'), or in all allout buffers if you customize the setting (eg, 'customize-variable').

This new version is also has a bunch of other fixes - see the link to the code, above, and [no longer present - see the respository source change log, instead] AlloutNEWS and AlloutChangeLog for details about the changes.

(organization of auto-fill functionality in allout was complicated by the hanging-indent requirements. there may be a cleaner way to do it, particularly since many things in emacs in general have gotten cleaner. if this new config var is enough to address the problem you noticed, though, i doubt i'll pay much more attention to that. i'm working on other allout stuff - soon there should be automatic escaping to enable including item-prefix looking text inside of item bodies without breaking the bodies, and i'm working on a whole nother layer, including nice graphics but more importantly, substantial cross references and settings, for the longer term.)

i have no way to reply to you, since i don't have your full name or email address - if you visit again and find this response, please let me know whether or not the change does what you need. enjoy!

From: klm, Date: Fri, 07 Jul 2006 16:41:47 -0400
Subject: auto-fill, mode-deactivation cleanup, more improved

i've finally reworked the cleanup provisions on allout-mode deactivation, so that variables, hook functions, and so forth are nicely reverted, which they weren't being before. (the new version of the mechanism is also much more programmer friendly, preventing overlooking some resumptions among other things.)

one specific thing that's done more nicely with the new mechanism is auto-fill provisions, so that auto-fill-mode is not activated if it wasn't already active. turning on auto-fill along the way, however, continues to provide auto-indentation that respects topic headlines, if allout customization settings are configured for that.

there are also now some unit tests, specifically for the modal-settings resumptions code. developers can select having the unit tests run automatically on module load by customizing allout-run-unit-tests-after-load.

i've uploaded the new version of the code, along with a new NEWS and ChangeLog files.

From: tom, Date: Sat, 11 Nov 2006 17:46:11 -0500
Subject: C-mode fixes submitted

I can't see a proper place to submit this, so I'm just submitting it here. Frustrated with allout's difficulty with C-style comments, I fixed the problem as I see it, and am submitting diffs which I hope will be included in the next version of allout.el

Specifically: It now understands about "*" in comment-start, and it now writes comment-end after a line.

[Begin diff]
696,697c696,697
< (setq outline-regexp (concat "\("
< (regexp-quote outline-header-prefix)
---
> (setq outline-regexp (concat "\(\"
> outline-header-prefix
2851,2852c2851
< " "
< comment-end))
---
> " "))
[End diff]

From: tom, Date: Sat, 11 Nov 2006 17:47:41 -0500
Subject: and that diff got horribly munged

Sorry about that, couldn't be helped.

From: klm, Date: Wed, 15 Nov 2006 20:02:06 -0500
Subject: C-mode fixes submitted

thanks for the patch, tom. (the formatting was preserved in the page source, it's just mangled in rendering.) it turns out that the problem was repaired in a more recent allout version - your patch is against a version that's at least a year old. download the newest version from the link, above, and give it a try...

From: jesse, Date: Fri, 19 Jan 2007 14:45:34 -0500
Subject: relation to linkd?

Hi Ken,

There's a new emacs package in town: linkd, see http://dto.freeshell.org/notebook/Linkd.html . What's the relationship between these two packages? Can one use both of them simultaneously?

Jesse

From: klm, Date: Tue, 20 Feb 2007 15:14:04 -0500
Subject: relation to linkd?

just took a look at linkd, and it seems like the linking stuff should work with allout, but use of the linkd block features may be problematic in allout, because there's no structural coordination between the two modes. that is, linkd blocks are created without regard to allout topic boundaries or containment structure. i don't think linkd was created specifically with allout in mind, and i didn't know about linkd until your mention of it.

From: tassilo, Date: Tue, 10 Jul 2007 17:08:15 -0400
Subject: Toggling the hidden state of the current entry

Hi Ken, I really enjoy allout mode, but one thing I always missed was toggling the hidden state of the current entry. I came up with this function which does exactly this:

(defun th-allout-toggle-children ()
  "Show or hide the current subtree depending on its current
state."
  (interactive)
  (save-excursion
    (allout-back-to-heading)
    (if (allout-hidden-p (point-at-eol))
        (progn
          (allout-show-children)
          (allout-show-entry))
      (allout-hide-current-subtree))))

Currently I'm waiting for assignment papers for GNU Emacs, so if you want I can provide a patch soon.

From: tassilo, Date: Tue, 10 Jul 2007 17:24:28 -0400
Subject: Toggling the hidden state of the current entry

Ups, that didn't work. Here's a paste: http://paste.lisp.org/display/44307

From: anonymous, Date: Thu, 06 Sep 2007 23:24:00 -0400
Subject: Add real keymap for hotspots?

Is there any chance of replacing the hotspot key mapping logic with a real keymap? I would have liked to be able to define other hotspot keys - specifically, I wanted to add "@" to work as "C-c @". But without a keymap to define a key into, there was no reasonable way to do it. If I submitted a patch replacing that logic with a keymap, would there be any interest in it?

From: klm, Date: Fri, 07 Sep 2007 02:01:23 -0400
Subject: Add real keymap for hotspots?

instead of retrofitting these things to allout, i'm working on a fairly thorough widget-ish allout addon, allout-widgets.el. it's described briefly in the main body of this page. the addon includes, among other things, using keymaps for the various active areas. i've made some progress since the version was last uploaded here, but it'll be a few days before i can upload the latest version - the guts are currently on the table. (i was actually editing it, working out a bug, when this comment came in.)

the addon's enhancements include graphical rendering of the outline structure, use of keymaps for the various outine parts with (as yet, mostly unused) mouse bindings, and undo that applies to structure exposure changes as well as edits. (the latter is necessary so that edit changes are visible as they're undone - settling a problem with regular allout.) there may be other features i'm forgetting, and in not too long i hope to get to dealing with multiply located outline items, to deal with things like layered topic versioning, alternately organized views of collections of hierarchies, and other database-y uses of outlines.

though there's still a ways to go to get it where i want, it's a pretty striking improvement on allout as it currently stands, if only for the graphical outline structure. though it doesn't yet surface some existing allout features - notably, distinctive bullets are currently hidden - i've been using it for almost all of my outlining for many months now, and wouldn't do without it.

From: klm, Date: Fri, 07 Sep 2007 10:24:39 -0400
Subject: Toggling the hidden state of the current entry

sorry about the delay replying - i will incorporate your toggle-visibility function. (i notice that the progn becomes unnecessary when you switched to using allout-show-current-subtree in the version you pasted to paste.lisp.org.)

From: tehom, Date: Fri, 07 Sep 2007 23:14:44 -0400
Subject: allout-widgets

allout-widget sounds fascinating. Tantalizing might be a better word, actually.

I was about to start coding smarter links as an add-on to allout, a "smarter C-c @", so to speak. The docs of allout-widget say "provides for situating and otherwise cross referencing items within and across outlines". But I don't see code for that yet.

Now I'm wondering whether I should go ahead and write it, or wait, or try to co-ordinate with you.

Have you considered what should happen if target files are renamed?

Tom Breton (Tehom)

From: klm, Date: Sat, 08 Sep 2007 16:07:12 -0400
Subject: allout-widgets

> allout-widget sounds fascinating. Tantalizing might be a better word,
> actually.tantalyzing for me, too. i'm only very slowing getting to the meat of what really interests me here.
> I was about to start coding smarter links as an add-on to allout, a
> "smarter C-c @", so to speak. The docs of allout-widget say
> "provides for situating and otherwise cross referencing items within
> and across outlines". But I don't see code for that yet.
>
> Now I'm wondering whether I should go ahead and write it, or wait, or
> try to co-ordinate with you.
>
> Have you considered what should happen if target files are renamed?

i'm not sure how to answer your question, though i do think that results from my efforts in this regard are going to be slow to arrive. that said, it may be helpful to know what i'm ultimately thinking about, to inform whether and what you do. i'm putting that in Allout Stable References, but be warned, it's very rough-draft and speculative (in american idiom, "pie in the sky"). that you're tantalyzed by the hints i've mentioned thus far suggests that you might find more details interesting, in any event. i also have a somewhat simplistic description of my ultimate interests, in Turning Answers Into Stories

i'm glad to hear of your interest!

ken

[P.S. incidentally, tom, i just learned about regress.el, on the emacs wiki, through it's association with your name. one of the last things i need to do for allout-widgets before i work on settings (and after i post an updated version for others to play with) is better infrastructure for regression testing. i hate being without test coverage where it's really needed, and have some rudimentary provisions in both allout.el and allout-widgets.el - look for -run-unit-tests-on-load. regress.el may be just what i need...]

From: klm, Date: Thu, 13 Sep 2007 17:40:12 -0400
Subject: new version of allout-widgets uploaded

i've uploaded a new version of allout-widgets.el which may be interesting to some allout users out there. there's still some missing allout features, notably, distinctive bullets are hidden - not removed from the text, just not yet surfaced to view. some fixes depend on a new version of allout.el i've also uploaded. give it a whirl, if you're interested, let me know if you encounter problems - i may have some time to polish things.

From: klm, Date: Thu, 20 Sep 2007 14:31:41 -0400
Subject: new version of allout-widgets uploaded

for any of you interested in allout widgets, i botched the version of allout-widgets.el that i uploaded previously. i just uploaded a new version that rectifies the mistake. unpack it in a directory on your emacs load-path, byte-compile (optional, but useful for large outlines) and load allout-widgets in your init somewhere.

(note that the allout-widgets.el file, as well as allout.el, have settings in their emacs file variable section to inhibit allout widget mode in the files, themselves. you can disable that inhibition by removing or relocating the line that assigns the allout-widgets-mode-inhibit variable.)

From: tehom, Date: Thu, 20 Sep 2007 21:16:55 -0400
Subject: Re: new version of allout-widgets uploaded

Thanks for the upload.

Some comments:

I like the item settings. I'm more excited about them than I am about widgets.

I had written a little code towards a similar idea. I had put the values in an s-exp immediately following the bullet. But I like your design better. It's more consonant with the text nature of these sort of files.

As the code stands, I can't really experiment because allout-get-item-widget just returns nil and the other interesting functions all want a widget as an argument. The problem may lie in how I invoked it, but I tried both allout-widgets-mode and allout-mode and then allout-widgetize-buffer.

I see there is code to read them in allout-assimilate-body-settings, but it assumes that widgets are set up and working. That makes it difficult for me to use it right now.

So if you don't mind, I'd like to send you changes I've written. It splits the non-widget code out of allout-assimilate-body-settings. I created some test example files and some tests - seems like continued lines and null settings are not working yet, so that might be handy.

One thing I've found helpful in writing tests for allout is to use outline-flatten-exposed-to-buffer to test that exposure is done right. I use this in conjunction with my mockbuf.el and a test-helper function I can send you.

When i want to test folding I make two example files, "EXAMPLE" and "EXAMPLE-folded". "EXAMPLE" is text in the configuration of interest, with "interesting" points marked by arbitrary unique strings. "EXAMPLE-folded" is the same thing minus any text that should be invisible.

(with-current-buffer
       (find-file-goto-text "example-file" "that-unique-string" ade:th-test-subdir)
       (outline-show-all)  ;;Undo any previous hiding
       ;;BODY: Do whatever is being tested, confident that it's
      ;;executed in "example-file" with point where the string
      ;;"that-unique-string" is
       (ade:th-compare-exposed-to-file "example-file-folded"))

(As I write this, I realize I could fruitfully encap that logic so that it only needs BODY and some parameters)

PS: Glad you like regress.el I have an updated version of it available called rtest.

From: klm, Date: Fri, 21 Sep 2007 00:13:13 -0400
Subject: Re: new version of allout-widgets uploaded

> I like the item settings. I'm more excited about them than I am about widgets.

interesting.

> As the code stands, I can't really experiment because allout-get-item-widget
> just returns nil and the other interesting functions all want a widget as an
> argument. The problem may lie in how I invoked it, but I tried both
> allout-widgets-mode and allout-mode and then allout-widgetize-buffer.

i'm not surprised that you're having some difficulty getting it going - you're probably the first person besides me to try it. in fact, there's a priming measure that i forgot to mention, in the docs and on this site. you need to do (allout-mode-widgets-init) after the allout-widgets code is loaded. i need to include a note about this step prominently in the file commentary, and in some main entry function.

also, allout-get-item-widget's behavior is a little odd. it only succeeds when point is located in the structure area of the item (before the body text - it's explained in the function's docstring). this can be beguiling.

one thing i want to do for testing is to have various outline static conditions, like text-property locations, recorded in emacs local file variables for comparison during tests. a regression testing harness should help make that fairly easy to automate.

From: tehom, Date: Fri, 21 Sep 2007 16:13:08 -0400
Subject: Re: new version of allout-widgets uploaded

Thanks for the directions on starting widgets. I'll definitely give it another whirl. Yes, when I did that, it worked, and graphics appeared in response to the outline.

allout-get-item-widget's behavior ... only succeeds when point is located in the structure area of the item

Maybe save-excursion, move point and get the widget? This seems to work as a stopgap measure:

(save-excursion
(allout-beginning-of-current-entry) (allout-get-item-widget))
i think the way i want to do regressions is to make some routines that record various conditions of an outline - where the text properties are, for instance - in file-local-variables using the routines, and have the tests

...test that the result is the same as before? That's one way to test, and it's tempting if the code already works. For testing output that has the property that if it looks right it's right, that sort of testing is perfect. But tests can do more for you than that. I find that test-driven design does wonders for me, but one needs to design and write the testability functionality first.

Tom Breton (Tehom)

From: klm, Date: Sat, 22 Sep 2007 16:07:27 -0400
Subject: Re: new version of allout-widgets uploaded

> ...test that the result is the same as before? That's one way
> to test, and it's tempting if the code already works. For testing
> output that has the property that if it looks right it's right, that
> sort of testing is perfect. But tests can do more for you than that.
> I find that test-driven design does wonders for me, but one needs to
> design and write the testability functionality first.

you're entirely right, and i have enjoyed programming that way - particularly when developing substantially layered applications. i've found it hard to do, though, when graphical UI interactions were an essential part of the functionality being tested. in fact, that concern partly explains the focus of my comment - i realized that i could capture many of the sensitive aspects of the allout widgets UI by recording and comparing the character (text and overlay) properties positions, detecting discrepancies that way. it so happens that i'm finding the way i'm using properties brittle, so i'm a bit obsessed with that. (i think i've figured out a way around that, and intend to make a substantial change in the way that item body regions are identified, avoiding using character properties for that.)

none of that addresses a major concern, for which i had already started to create a little bit of testing infrastructure - settings. i'm thrilled that you're interested in that, to the point of sending some tests that look like they'll be quite useful, and maybe help get me kick started - yay!

and, i've uploaded another version of the allout-widgets with some more mistakes polished off - plus, some of those setup notes that were missing.

From: tom breton, Date: Sat, 06 Oct 2007 17:54:34 -0400
Subject: Let's move item settings to before item body?

Ken: I've been working with allout-item-settings a fair bit recently. I have some code that reads and writes them - not to and from widgets, though. I sent you an early version of it.

I've encountered what I think is a serious problem: When an item's body is split into subtopics, the settings end up in the last child, not in the node itself (as new parent). This happens because the settings follow the body.

I think the best solution is to move the settings to above items' bodies instead of below. I'd like to implement this.

Tom Breton (Tehom)

From: klm, Date: Tue, 26 Feb 2008 16:24:59 -0500
Subject: allout.el xemacs compatibility

on prompting from someone who needed it, i've finally spent a bit of time getting the recent allout.el working in the recent version of xemacs. there are still some small glitches, but it seems quite usable. (allout-widgets.el is another story. i'll have to address differences in graphics/images instantiation, and probably won't do that immediately.)

Date: Sat, 15 Aug 2009 04:50:52 -0500
Subject: Cycling like in org-mode??

Is it possible to have cycling akin to that in org-mode (Show-children->Show-subtree->Hide-tree)?? Currently allout-toggle-current-subtree-exposure only toggles the hidden state of a tree. Thanks.

From: klm, Sat, 15 Aug 2009 10:00:18 -0500
Subject: Cycling like in org-mode??

> Is it possible to have cycling akin to that in org-mode
> (Show-children->Show-subtree->Hide-tree)?? Currently
> allout-toggle-current-subtree-exposure only toggles the
> hidden state of a tree. Thanks.

"possible?": yes. "already implemented?": no. "hard to implement?": probably not. i like the idea, and will try to find time to implement it, but i already have a backlog of allout things pending (some pending release, like user-customizable keybindings), so i can't say when i'll get to it. i can encourage you to look into implementing it yourself - but if you're not familiar with emacs lisp coding, allout code is not the easiest place to start