Rob Reed

Tuesday Feb 19, 2009

• Ode is written in Perl.
• The content of an Ode site is a collection of plain text files.
• The look and layout of a site is entirely determined by 'themes', which are no more than standard (X)HTML and CSS.
• Ode uses the filesystem as its content database.
  • This more than any other design decision impacts how Ode is written, how it operates, and how it's used.
• Is that all? No.
  • Ode is extensible.
    • Only essential behaviors are implemented in the script itself.
    • It's possible to extend what Ode can do by taking advantage of an 'addin' scheme which allows for new features or even redefining built-in routines.

Ultimately, Ode should support all of the key features found in other personal publishing packages for the web, including:

• Permalinks
• Date-based browsing and archiving
• Commenting
• Syndication (feeds)
• Local search
• In browser editing (with live preview)
• Presentations (conforming to the S5 format)
• and many others

In fact the majority of these features are already implemented.

Because content are plain text files and Ode uses the filesystem for its content database, negotiating an Ode site amounts to:

1. Editing
2. File management
3. File transfer

When taken together with the wealth of software available for these tasks, Ode is a powerful and flexible little tool.

Editing

• You're free to use any editor you like!
• There are many fantastic editors available, some of the best are open source apps and of course there are also some great commercial applications.
• These editors are as varied as the people who use them and the jobs they're asked to do.
• As a class, editors are some of the most mature, subtlety brilliant software applications available.
  • Why then, when it comes to writing online, are we asked to abandon these finely honed tools in favor of the blunt instrument that is the HTML form?
• Ahh, but being able to write in a browser is so much more convenient. I agree.
  • We can do that too. :)

File Management

• Posts are files.
• Directories are categories.
• Assuming you've used a computer in the last approx 40 years, those two statements are enough for you to understand how to manage content for an Ode site.
• But just like managing and organizing other types of files, this scheme proves to be robust.

Create any number of categories and subcategories in any arrangement that works for you.

Examples:

/Review/
/Review/Movies/
/Review/Software/
/Review/Software/Open_Source/

/Tufts/Courses/Comp250SA/

Here's one you'll never need to do:

/2008/
/2008/02/
/2008/02/06/

Ode handle dates like this automatically.

File Management continued

Static websites work like this too, but because Ode is not static we can take better advantage of this arrangement.

For example:

• Themes can be defined and redefined at any level of the hierarchy.
• Any category or subcategory can be syndicated and subscribed to independently.
• Posts can be reorganized without affecting their date-based permalinks.

File Transfer

Interacting with a remote server is no more complicated than file transfer, which allows for a great degree of flexibility.

You're free to use:

• SFTP
• SSH
• WebDAV
• HTTP

...or any other file transfer/data exchange protocol that works for your environment.

Other

There are other advantages to keeping content as discrete plain text files related to:

• Scripting and automation
• Synchronization among multiple installations, etc.
• Migrating data into and out of Ode
• etc.

'Ode is simple'

• With the stipulation that
  • 'Simple means you know how it works.'

That assertion requires that the documentation be done well.

Ode should be lightweight.

Resource requirements should be minimal.

• By taking advantage of services already provided by the OS, other freely available software, and well-established protocols, we can minimize the overhead involved in running this application while improving its outlook in terms of reliability, security, interoperability, etc.

There should be a minimum of internal and external dependencies.

External Dependencies

• Prerequisites should be eliminated to the extent that it is reasonable to do so.

Internal Dependencies

• Projects that encourage group participation and present a modular architecture can quickly run into problems with conflicting competition and dependency among components.
  • For example, as an end user I want to install an extension but can't because it requires an extension I don't want or can't install.
  • Ode should be designed to avoid these these types of problems.

The code should be accessible for people new to programming.

• The source should be well-documented so that it is accessible to the widest possible audience.
• The code should be clear not clever.
  • Anyone trying to demonstrate their programming prowess would be better off starting with a different project. :)
  • The code should be written so that it is:
    • Intuitive and obvious to experienced programmers (so as to encourage their participation).
    • Accessible even to very new programmers (so as to encourage their participation too).

The code should be accessible for people new to Perl.

Perl's strengths are also some of its weaknesses:

• It is a permissive language, and so has a tendency to result in unstructured programs
• It is overly idiomatic.

For these reasons Perl's critics often deride it as a 'write-only' language.

There's nothing about the language that prevents code from being well-written:

• Perl's weaknesses can be overcome by adhering to good coding practices.
• The style of the source code should be as explicit and language-neutral as possible as opposed to being terse and overly idiomatic, except where the idioms are more efficient.
  • Wherever idioms are preferred, liberal commenting should be used so that people who are unfamiliar with the eccentricities of the language are not lost.

The project should not use a separate package manager which:

• Adds an external dependency.
• Potentially fractures support and maintenance issues between multiple projects.
• Complicates initial setup and configuration.
  • Especially for users new to the package manager who must, at the outset, contend with multiple new projects.

Finally, it's unnecessary because managing Ode's packages (addins) is straightforward.

Work locally or remotely

• Manage content
  • Create
  • Post
  • Edit
  • Share
• Design themes
• Write and test addins

It should not be necessary to introduce mulitple 'modes' of interaction.

It should be possible to synchronize local and remote instances of the same site.

The entire platform should be portable.

• Ode should run, essentially unmodified, on all of the mainstream desktop and server operating systems, including:
  • Linux
  • Mac OSX
  • Windows
• It should be flexible enough to take advantage of best in class or user preferred apps and services available for each platform.

• Presentation or Post?
• Edit Presentation Content Just Like Posts
• Design Presentations Using Themes
• Create, Share and Present
• Collaborative Presentations
• How It Works

• Presentations are posts (and vice versa)

• Create or modify presentations using a web browser
  • Fix mistakes quickly
• or with a text editor
  • Use a comfortable, familiar application
  • Without the bloat and quirks of presentation apps
• Write in either html or markdown

• Treat the design of your presentations like any other theme
• A theme in ode is a bundle (folder) containing a page "template", CSS file(s), an image directory maybe, and whatever else you want to include
• All we need to do is add the S5 javascript files
• Presentation design involves XHTML and CSS

• Create a post about a particular topic or idea
• Then switch themes to present information from that post to a group
• Create a presentation and ask for comments
• Or even invite others to edit the presentation directly
• Share presentations as plain text (email, IM or the web)

• Controls are:
  • Next slide: Space bar, return, right arrow, down arrow, page down, click anywhere in slide that isn't in the control area (lower right corner), or use the control area
  • Previous slide: Up arrow, left arrow, page up, or use the control area
• To invoke the navigation menu: mouse into the lower right corner of the slide

Rob Reed

Tuesday Feb 19, 2008

• Presentation or Post?
• Edit Presentation Content Just Like Posts
• Design Presentations Using Themes
• Create, Share and Present
• Collaborative Presentations
• How It Works

• Presentations are posts (and vice versa)

• Create or modify presentations using a web browser
  • Fix mistakes quickly
• or with a text editor
  • Use a comfortable, familiar application
  • Without the bloat and quirks of presentation apps
• Write in either html or markdown

• Treat the design of your presentations like any other theme
• A theme in ode is a bundle (folder) containing a page "template", CSS file(s), an image directory maybe, and whatever else you want to include
• All we need to do is add the S5 javascript files
• Presentation design involves XHTML and CSS

• Create a post about a particular topic or idea
• Then switch themes to present information from that post to a group
• Create a presentation and ask for comments
• Or even invite others to edit the presentation directly
• Share presentations as plain text (email, IM or the web)

• Controls are:
  • Next slide: Space bar, return, right arrow, down arrow, page down, click anywhere in slide that isn't in the control area (lower right corner), or use the control area
  • Previous slide: Up arrow, left arrow, page up, or use the control area
• To invoke the navigation menu: mouse into the lower right corner of the slide

Rob Reed

Sept 30, 2009

"Mine is an architecture documentation project"

• That's still true, but the emphasis and scope of what I'm doing is significantly different.

I started with the goal of completing what I called a 'case study'.

• Writing the architecture documentation for another project of mine (Ode).

Something was missing

• I didn't know how to apply the theory to my existing project.
• More generally, I was missing an approach to applying what I believe is a strong theoretical framework to the job of putting together the architecture documentation for any one specific project.

What I came up with is a general model for writing architecture documentation.

Early in the semester we were asked to read the prologue to the book "Documenting Software Architectures: Views and Beyond", which presented this list of 'Rules for sound documentation':

Note: This is review, what is new is the question...

Documentation should:

• Be written from the reader's point of view
• Avoid unnecessary repetition
• Avoid ambiguity
• Use a standard organization
• Record rationale
• Be kept current but not too current
• Be reviewed for fitness of purpose

Are these rules enough?

Again, from the book "Documenting Software Architectures: Views and Beyond" we have this definition

Architecture is what makes the sets of parts [of a system] work together as a successful whole. Architecture documentation is what tells the developers how to make it so.

We are told that architecture documentation should be:

• Abstract -- to be quickly understood
• Detailed -- to serve as a blue-print for construction
• Informative enough -- to serve as a basis for analysis
• Prescriptive -- prescribing what should be true
• Descriptive -- describing what is true

And that it should serve as:

• A means of education
• A primary vehicle for communication among stakeholders
• The basis for system analysis

The "IEEE Recommended Practice for Architecture Description and Software-Intensive Systems" (IEEE Std 1471-2000)

presents four scenarios "intended to suggest the range of uses of the recommended practice"

1. Architecture of single systems which deals with new system construction Note: This is the situation where we want to create the architecture description for a system which does not yet exist and then use the description to guide development
2. Iterative Architecture where the architecture, delivered systems, and stakeholders co-evolve In this case the architecture description and the architecture are constructed together such that the current system is described by the current documentation both of which may change going forward.
3. Architecture of existing systems which is relevant when there has been development without a corresponding architectural description Note: What's described here is the situation where we have a system, and so an architecture, but not the architecture documentation which either doesn't exist or can't be used (because it can't be found for example) so we work backwards to build the description from the architecture itself, after which we can use the new documentation to reason about and maybe decide that we need make changes to the architecture.

This is the situation that describes what I proposed to do, that create the architecture documentation for an existing system.

4. Evaluation, the purpose of which is to include the quality of architectural description and predict the quality of systems based on that description. Note: The key ideas here is evaluation. We evaluate the quality of the documentation and based on that evaluation make judgements about systems based on the documentation.

Requirements

• We must be abstract and detailed
• 'informative enough'
• prescriptive and descriptive
• allow for education, communication, and analysis,

Accomplishing all of this is no small feat.

These scenarios...

• speak to the importance of architecture documentation as a persistent influence.

• suggest that architecture documentation can be introduced at any point in the life cycle of an architecture.

All of these figure prominently into my approach and the resulting model.

Note: We have been introduced to many ideas about software architecture and architecture documentation throughout the course of the semester.

And there are many details that fill out the broad conceptual framework, but here is my list of fundamental concepts... without which we cannot hope to do an adequate job with architecture documentation. (followed by the contents of the slide)

This is my list of critically important concepts:

• The concept of the view;
• The role of architecture in suppressing information not relevant to the task at hand;
• A commitment to representing the interests of stakeholders;
• The idea that the architecture documentation and the architecture itself exist and are maintained together throughout the entire life cycle of the architecture;
• The importance of considering elements which are included as part of the architecture and its environment, at any point in the life-cycle. And to do so in such a way that these elements are fixed (i.e. dependable or unchanging) with respect to the architecture.

If one thing is clear from what we've discussed in this class it's that the work we set out to do is difficult and it isn't getting any easier.

For all of the problems we had with the autonomous computing paper, here is a quote from the article that no one was bothered by enough to refute:

Computing systems' complexity appears to be approaching the limits of human capability, yet the march toward increased interconnectivity and integration rushes ahead unabated.

Though not the answer to all of our software complexity woes, we can

Note: If we're willing to entertain the idea that autonomous computing is a potential long term solution to the complexity problem, and if the authors of that paper are correct that the realization of their vision will require

"a concerted, long-term, worldwide effort by researchers in a diversity of fields"

We had better learn to tolerate complexity

by improving the tools and techniques we use to document software systems.

Note: I'll warn you that there are several slides dealing with problems associated with documentation because

1. 1. There are a lot of problems.
2. 2. We must be familiar with the problems if we hope to develop a successful approach

I have no doubt that we'll all recognize these problems so it shouldn't take long to get through them...

Easier doesn't necessarily mean easy.

We have few well established patterns for dealing with documentation.

There are many questions that must be answered before we can hope to do an adequate job with architecture documentation.

• Where to begin?
• How do we manage this wealth of documentation that is being, or has already been generated by others involved in the system?
• How do we describe the documentation itself (i.e. how do we document the documentation)?

These are not the only challenging questions.

The existence of multiple documents

• Risks fracturing the attention of people involved in the system;
• Allows for the possibility of parts of the documentation disappearing;
• Can lead to disagreement or confusion about what has been done and what still needs to be accomplished.

It is not uncommon to find ourselves in a situation where we have multiple and competing documents, or multiple versions of the same document.

• Leads to misunderstanding, misrepresentation, and ultimately architectural confusion (i.e. confusion about what the architecture is).

A one document approach may be interpreted as a single task.

• The work may be assigned to one person or group.
  • May require an unreasonable amount of time and effort to put together
  • Documentation is often either neglected or passed along to a separate group, who are not necessarily the best people for the job.

Writers of technical documentation are often detached or isolated from the development groups.

• Often leads to misunderstanding and inconsistencies between the documentation and the implementation itself.

Although the architecture documentation should prescribe how the system should work, if it does not adequately describe the current behavior, it may be regarded as irrelevant

• This may instead describe a broken implementation which doesn't agree with the intended architecture.

We are often unrealistic about the investment required and as a result do not set aside adequate resources for documentation.

The ability to write documentation well is a skill that not everyone involved in the development of software possesses, or practices.

• The emphasis of documentation in education, or lack thereof, may be an important factor.

We said early in the semester that software development is essentially linear, and that this may contribute to our inability to properly specify complex architectures.

Note: Taking into consideration modern development environment, and techniques, hundreds and thousands of developers working on the same system simultaneously and independently, and collaborating with teams that may be separated gegraphically, I don't know that it's true that software and software development is strictly linear.

• Documentation has remained stubbornly linear (whether or not this is true of software development)
• Even with architecture description languages, (a la UML), our diagrams are typically one of a linear set of ideas bound together that must be considered from beginning to end to be understood.

Lastly, we may want to question some of the the assumptions that underlie the stated goals of architecture documentation.

• Is it appropriate to expect any one person, or small group, to anticipate or be capable of adequately representing the unique and often divergent concerns of all stakeholders?
• Do we expect the architect to write the architecture documentation?
  • Is this reasonable given the unsettled role of the architect?

How might we address some of these problems?

In my estimation must be able to:

• Compose documentation (i.e. produce many views of a single document);
• Note:To minimize problems with either single and multiple document approaches and afford the flexibility necessary to allow for the flexible definition and representation of views.
• Bridge architecture and implementation level documentation;
• Note: To avoid the sort of architectural confusion that results when the architecture description does not agree with implementation level details.And to expose these inconsistencies, something which is important to the architect - even if the the details of implementation alone are not.
• Minimize the effort required to produce the architecture documentation by incorporating work which has already been done;
• Establish a connection between the the architecture and the architecture documentation;
• Involve everyone in the documentation effort, such that the collected knowledge about the architecture is reflected in the documentation;
• Explore nonlinear approaches to documentation, so that we can preserve detailed knowledge about the architecture without exposing an overwhelming amount of information;
• Pursue an approach that is flexible and adaptable enough to accommodate the potentially unique and varied concerns of existing stakeholders as well as meet the needs of new stakeholders.

My approach to the project was straightforward.

I started early in the semester with the work I knew how to do, not knowing a lot about architecture concepts and architecture description at the time.

• This involved producing and pulling together a wealth of documentation.

Note: Literally hundreds of pages of documentation for ode alone not including other elements of the environment (libraries, services, protocols, etc.)

Next, I identified what I considered to be critical concepts in the current thinking about architecture documentation.

Finally, I needed a method which would allow me apply the concepts.

• What I came up with, and what I've been working on since, is a general "model" for pursuing architecture documentation.

Borrowing concepts from component-based software engineering, we create a set of documentation components

These include a wide variety of architectural elements related to the architecture

Anything we want to

• represent in one or more views of the architecture documentation.
• describe in greater detail

Each documentation object is essentially a package of progressively more detailed versions of the documentation for one element of the architecture.

Imagine that all of these documentation components arranged in a grid

• That grid, which I'm calling a documentation component browser, defines the scope of the architecture.

Note: I am aware that this slide takes a turn toward the discussion of a possible implementation but it is a useful mechanism/vehicle to help understand how this model might work (I hope) and this is where I'm headed at the moment.

I don't lean too heavily on a discussion of the implementation, just this slide and the next.

The browser allows us to:

• Navigate any of the individual documentation objects
• Define, edit or delete documentation objects
• Filter the collection of documentation objects based on associated attributes or metadata
• Search over the entirety of the architecture documentation
• Select from a list of views to highlight only those components that are included in the view
• Create new views and assign documentation components to views

The browser does not show connections between documentation components.

• Doing so would violate the key concept that views hide documentation that is not relevant to the task at hand (and lead to confusion).

We need another interface for composing views.

When we select any view (from an collection of views available in the browser), The documentation components of that view are highlighted.

If we choose to compose the view the browser's grid drops away.

• Leaving only the components of the view
  • which we can then manipulate by connecting and rearranging them for example.

By connecting the documentation components, we specify relationships among the documentation, in a way that is similar to what you might think of doing with an abstract ADL.

• The purpose is to allow us to reason about a system that either we intend to build, or one which already exists.

A view is a physical document in the sense that it can be printed, copied and shared.

We define a single architecture document and use it to hide, selectively expose, and manipulate what may be hundreds, or thousands source documents.

• From this one document we can produce many more, each with a narrower focus, and each potentially tailored to the needs of a particular stakeholder.

The model supports two approaches to documenting relationships in the architecture.

1. We can include relationships among the documentation components, and then compose the components using only 'dumb' connections (which indicate nothing more an ordering over the documentation).
   • With this approach, we can take advantage of the expressive power of our components as packaged abstractions.
2. Note: to describe complex connections at whatever level of detail is required
3. We can also choose not include relationships among the documentation components and instead deal with them as part of composition...
   • in which case we need a more robust collection of graphical objects and tools for composing documentation components.

This is a decision for the architect

• If connections are very complex then we may choose the former approach.
• In an environment where a description language like UML is already commonly used, then it may be more appealing to deal with connections as part of the composition

Who are these stakeholders?

They may be roles, or individuals (i.e. specific people) involved with the architecture.

• There may be many such stakeholders, with narrowly defined concerns.

Is it possible to anticipate the needs of all of these stakeholders?

• No, I would argue that it's not.
• A better approach is to allow stakeholders to generate their own views that meet their unique needs.
  • in addition to a set of stock views which are composed by the architect herself.

They are reusable.

• Each is one of a library of components.
• Documentation components can appear in the browser, in one or more views, or part of another component's abstraction hierarchy.

Is everything a documentation component?

No. Consistent with other ideas about architecture documentation, there are both architecture and implementation level details.
• Architectural level details are represented as documentation components.
• Implementation level details are simply graphical, or textual descriptions of some part of a system.
  • These appear (only) as part of the abstraction hierarchy for some architectural element and lack details of their own.

The approach is intended to be nonlinear, flexible, and adaptive.

To this end it:

• Defines the scope of an architecture including the environment (as described by the IEEE recommended practice)
• Allows for composing documentation in response to the needs of the stakeholders
• Bridges between what is typically considered architecture and implementation level documentation.
• Is general purpose enough to work with many different styles of architecture.
• Emphasizes the key concepts of architecture documentation (already discussed).
• Is implementable and adoptable now.
• Does not introduce a lot of overhead beyond what is necessary to realize an architectural vision of the system.
• Is easy to understand.

Note: You may recognize that many of these advantages answer/address the problems of documentation discussed earlier and correspond the proposed solutions which I've also talked about earlier in this presentation.

The model is adoptable now

After reading the Rapide paper, the class talked about whether the strategy epitomized by that language is adoptable

• given the current culture and prevailing approaches to the job of software engineering.

Can we pursue architecture-based approaches without architects?

• I would suggest that the answer is no, and that this is a fundamental disconnect between what we suppose we must do to be successful, and what we are capable of achieving based solely on theoretical or technical progress.

In comparison, I believe that the modest idea I'm proposing is adoptable now because it reflects the developer-driven organizational style that currently dominates software projects.

• What's more, this model can serve as a bridge to a potential future when the field may be more architecturally driven.

Because it is a documentation model it does not place any restrictions on or make any assumptions about the system itself.

• A potentially promising solution may not be worth considering if it reduces an experienced development team to a group of novices.

Documentation can serve as a path toward architectural change.

• It's reasonable to think that an architectural approach to documentation may lead to interest in architectural approaches to building systems.

Documentation bridges architecture and implementation level details, serving as a medium for communication between stakeholders.

The model addresses common logistical issues related to maintenance of documentation.

• We are asking everyone involved in maintaining the documentation to do a reasonable amount of work, and to be responsible for parts of the system they are familiar with.

The model introduces only a minimal amount of overhead.

• The model takes advantage of implementation level documentation but does not burden the architect with implementation level details.

Information that is not included in the documentation takes on an active meaning.

Current work

I'm continuing to refine the model by building a 'prototype' that works roughly as described.

• This means that I'm cobbling something together to play with these ideas more.
• I expect that this prototype will be fairly limited in its usefulness, except that hopefully it will help me work out some of the details.

Future work

Ultimately the end result of this may be the development of a set of tools for creating, manipulating and presenting architecture documentation in a way that is consistent with the model as described.

Thank you