Just a quick note:

If you are testing a module which you include in multiple controllers you need to describe the controller name you are testing, not the module name or you end up with the stack trace below

 Failure/Error: Unable to find matching line from backtrace
 NoMethodError:
 undefined method `original_path_set' for nil:NilClass
 # /Users/macmacmacmacmacmac/.rvm/gems/ruby-1.9.3-p0/gems/rspec-rails-2.8.1/lib/rspec/rails/view_rendering.rb:113:in `block (2 levels) in <module:ViewRendering>'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example_group.rb:435:in `instance_eval'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example_group.rb:435:in `instance_eval_with_rescue'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/hooks.rb:45:in `run_in'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/hooks.rb:80:in `block in run_all'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/hooks.rb:80:in `each'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/hooks.rb:80:in `run_all'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/hooks.rb:368:in `run_hook'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example_group.rb:297:in `block in run_after_each_hooks'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example_group.rb:297:in `each'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example_group.rb:297:in `run_after_each_hooks'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example.rb:221:in `run_after_each'
 # /Users/mark/.rvm/gems/ruby-1.9.3-p0/gems/rspec-core-2.8.0/lib/rspec/core/example.rb:86:in `block in run'

Solution is nicely formatted here: https://gist.github.com/1869263

Aphro

I’ve been captivated by this topic and am currently in the process of building a Ruby gem to help consume HATEOAS compliant services.
I’ve called it Aphro for want of a better name and because I think HATEOAS isn’t a very nice name.
One thing I’ve been finding in my thought experiment is that the hypermedia constraint (better name in my opinion than HATEOAS), is great for discoverability and that with that discovery process comes an obvious performance cost. API consumers want concise API calls and wouldn’t expect to browse every time from the root to find the URI and required parameters. Now if you re-read that and imagine a human interacting with a web browser you have all these metaphors that you can just carry across into the API consuming world.

“An API consumer doesn’t want to make multiple requests just to find the required fields for a POST to a resource that probably hasn’t changed it’s URL” == “A user doesn’t want to browse from the Google homepage to go to their Gmail to compose an email” They can just save a link: https://mail.google.com/mail/#compose

Not only that, their browser could cache the page. Well your API consumer can do the exact same thing. What’s a good way for knowing the required fields for a POST to a resource? Save the webpage with the form on.

More traditional APIs with their pre-specified URLs just don’t have the discoverability part. In caching the pages you lose a guarantee that you have the correct form fields, and would need to get an updated version if necessary (E-tags and HTTP statuses can help here too).

I’m working on being able to cache API calls (read save webpages), so that you can have API interactions that do the discovery part, useful for initial development, and monitoring jobs to check your producer still works as you’d expect, but then also have simpler closer to traditional API calls that don’t have to do the discovery part. So once the cached calls are saved what would it mean for your client? You could do something like this:

#Discovery section and used in monitoring jobs
twitter = Aphro.client 'api.twitter.com'
twitter.sign_in username, password
twitter.tweet 'new message'
twitter.cache_directory= 'tmp/cached_twitter_api/'

tweet_api = twitter.cache_page :tweet
twitter.save_session :twitter

tweet_api.tweet

#_____New session - day-to-day client code
twitter = Aphro.open_session :twitter
twitter.tweet 'my tweet using the saved session'

What else could you do? I’m also working on auto-generation of consuming client code based on browser interactions. I’m interested in using the Browser as documentation and as the API. It could also be used as a way to dynamically generate client code. I think Selenium IDE might be positioned well as a means of recording interactions and generating client code.

Do we really have to use bulky XHTML?

I think with content-negotiation and a clear universal standard way of expressing forms in JSON then there’s no reason that we can’t have all this goodness with JSON too. I’ve not come across any clear standards just yet, and so I’m beginning to think that just writing good clients to consume forms and links might be the answer. In order for the idea to really take off the tools need to be in place and it needs to just be easy to use.
There either needs to be an easy way to output forms in a standard way in JSON, or a really good client to simplify dealing with forms from say javascript.

My main reason for preferring JSON over XML is that it is so much easier to read as a human being. The fact that there are less characters and that it is more efficient isn’t my primary motivation as a Rubyist. It’s that it’s easier for me the coder to read. Perhaps if I worked at Google where the cost of running the infrastructure and servers is much higher than the cost of the developers then it might make sense to consider optimising for the machine. But the main reason Ruby is useful to businesses is because it can reduce the cost of development by simplifying things for the human. And a lot of the time development cost is a concern for businesses.

So parsing XHTML as an overhead for the machine? I don’t think that’s so bad in a lot of circumstances. What do you get for the trade-off? The app is the API which is the documentation which is the basis for the monitoring jobs that your API consumers run which is the client code that your consumers run.

Still I think a nice JSON output format (with forms) could help win people over, even if I do find web pages easier to read than JSON.

I’m veering further from the simple vim and ruby path and may need to rename this blog.

I’ve been trying to gain insight into the meaning of HATEOAS.

This started off as a README for my fork of Steve Klabnik’s Hateos project.

I have an intuitive sense that there must be value in the principle of HATEOAS, but I’m really not fully sold on the ideas.

I’m posting the latest revision of my thoughts here as while not well crafted, it is quite suited to the blog format.

Why am I forking this repo?

• Because I think there’s mileage in exploring ideas. I’m interested in the direction the web is taking and the future of APIs and human and machine interaction.
• Because I want to braindump some of these ideas.
• Because I was going to grab the hateoas name for a rubygem but didn’t get round to doing it.

HATEOAS, JSON and forms

I like JSON, it’s readable and clear. It lends itself well to friendly formatted output it’s beautiful because it sits in a comfortable place for humans and machines. It doesn’t work well for HATEOAS. HATEOAS requires links for conveying information about where you can move to next and forms. Well you can’t really have JSON forms. Not without some predetermined way of expressing them. And that presumably counts as out-of-band communication according to Roy Fielding.

I’m not entirely sold on Roy Fielding’s ideas. I don’t feel that anyone has truly made them accessible to people like me and reading his blog posts doesn’t help a great deal. That doesn’t mean I think it’s wrong. It’s just that without explicit examples of exactly what a 100% REST compliant system is, it’s difficult to decipher Roy’s prose.

Trying to understand HATEOAS

I’d like to gain a full understanding of what something that is not just RESTful, RESTish or RESTesque, but actually REST. Roy Fielding does the community a bit of a dis-service by not providing positive examples of REST systems, only examples that fail to meet his expectations. To paraphrase:

“If you don’t have full HATEOAS compliance then you don’t get REST and you shouldn’t call it REST.”

OK.

Good. I’d like to create an API that is fully REST compliant. Where do I start?

I need to understand what HATEOAS is and how I can use it.

So it’s Hypermedia as the Engine of Application State.

A REST client needs no prior knowledge about how to interact with any particular
application or server beyond a generic understanding of hypermedia.

A generic understanding of hypermedia is a little vague.

from http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

^ This is a part that I am conceptually struggling with. It seems to me that there would always be coupling between a server and a client. By trying to remove the constraint of using out-of-band information you reduce the amount of pre-requisites but you lose something at the same time.

Communication is about expressing ideas and it would be interesting to create a means of communication between any two systems that requires NO PRIOR KNOWLEDGE of what that system is and/or does. Communication also depends on a common shared vocabulary. If I were to consume a twitter API then I would have prior knowledge of why I wanted to consume it. I have a huge amount of out-of-band information about why I should use it. I also might reasonably assume that I can POST tweets. But programatically navigating this API without knowledge of anything about it? Perhaps I’m misunderstanding but the idea of writing client code that is so flexible that the services it connects to could completely change boggles my mind. If I picture this scenario of ultimate future flexibility of decades long systems that can be so flexible because they are fully REST and HATEOAS compliant and can adapt to all sorts of changes because the client assumes nothing in advance then I can only picture at the other end some kind of human interaction.

Pseudo-interaction of a REST client:

* Visit api.twitter.com
* See list of links in headers for possible next actions
* Check for tweet rel < - how is this determined ? (no prior knowledge so how would I even know I might want to tweet?)
* It appears twitter no longer has a tweet action
* ...
* Perform alternative action (Who or what decided this)

So maybe we can write clients that explore APIs and report back to more intelligent agents about what is currently possible. Who are these agents? Intelligent software agents? Humans? Is this out-of-band communication yet?

HATEOAS seems akin to inventing a human communication protocol that requires only an introduction of two individuals on earth. Maybe this is a strawman but it sounds a little bit like two people dropping the prerequisite for speaking the same language. But perhaps this relates to the concepts of ‘standards’

A REST API should never have “typed” resources that are significant to the client.
Specification authors may use resource types for describing server implementation
behind the interface, but those types must be irrelevant and invisible to the client.
The only types that are significant to a client are the current representation’s media
type and standardized relation names.

‘Standardized relation names’ sounds to me like out of band communication. It may sound like I’m naively comparing each point to human language and communication but I think there are parallels.

It seems like it is confidently suggesting a group can come up with a set of standards that can summarize and categorize all future expected communications.

It sounds worryingly like a top-down approach to thinking about how communications occur. It reminds me of attempts to make languages like Esperanto.

An idealist would hope all humans can work towards creating a common language and that if we can create a common enough framework that can be expanded upon as necessary then we can allow communication from anyone to anyone else on earth.

But these approaches have so far failed. They mainly ignore the advantages of specialisation, the benefits of exclusivity within a small group and the efficiency of communication when new thoughts and ideas that might be relevant only to the local area can be expressed. Languages merge into one another and dialects merge into one another. Each language has a wealth of predetermined pre-encoded information that is specific to the culture of the speakers of that language. There are optimisations for expressing more common things more easily and I don’t doubt that certain languages lead and guide the speakers to communicate in ways or about topics that lend themselves to easier communication in their tongue.

The gist of what I’m saying probably sounds thoroughly disconnected from what REST is or aspires to be, but I’d like to express the parallels. I don’t think standardised out-of-band communication could work. I think out-of-band communication and state are necessary to have expressive and meaningfully useful communication.

Out-of-band and state have parallels with pre-learned human language, human memory and interactions.

I think that for machines to have meaningful relationships with each other they may need to have a wealth of pre-determined concepts to build upon and expand upon. I also think that at present the quickest way to express this information in a way that enables humans and machines to interact well together is for humans to have discussions, ideas and inspiration and to communicate how these ideas might be shared between machines by agreeing upon simple meaningful APIs.

I may well be missing the point, but I think coupling in communication is possibly a necessity for meaningful communication. Client and server like speaker and listener are inherently bound together and without shared prior knowledge machines can no more have meaningful discourse than a native Mandarin speaker who speaks no other language can have a deep and meaningful discussion with a native Spanish speaker who speaks no other language. And any simple prescriptive predetermined protocol that the two might agree upon is still bound to failure.

RDF

I need to understand RDF as it might give more insight into how XHTML APIs could utilise more human input and actually meet the HATEOAS constraint.

RDF primer http://www.w3.org/TR/xhtml-rdfa-primer/

Summary

This README has turned more into a blog post and brain dump. It’s not particularly well formed, but I’m interested in exploring my ideas further and seeing where I end up. I’d like to find that by the end of this exploration I understand HATEAOS and REST fully and I’d be happy to have my doubts quashed by a crystallising epiphany about what HATEOAS truly means.

I’ve benefitted greatly from utilising principles of ‘REST-ish’ design (sorry Roy) and so I’m not about to cast Roy Fielding and his thesis off as emperor’s clothes troll-tastic material just yet. But I am prepared to mention my doubts.

The ultimate troll would be if the complete conclusion of a 100% REST compliant system is that of a user interacting with a web browser and reading the information presented to them by first visiting the ‘homepage’ of the site.

API Exploration

Without fully understanding the benefits of a HATEOAS compliant system I’d like to consider what I might expect from using a client API:

twitter = Hateoas.site "api.twitter.com"
twitter.actions
# => [sign_in: {method: :get}]
twitter.sign_in user: "mark@example.com", password: "password"
twitter.actions
# => [read_messages: {method: :get}, tweet: {method: :post}]

Now the main gist of my point comes down to the prior knowledge of the system

Looking at the example again:

twitter = Hateoas.site "api.twitter.com" #agreed prior knowledge of URL
twitter.actions    #generic - no prior knowledge of twitter API. (actions method could be a generic method defined on the API but it is not preagreed information between client and server)
# => [sign_in: {method: :get}]
twitter.sign_in user: "mark@example.com", password: "password"   #choosing to call this is prior shared knowledge

Now HATEOAS will protect us from URL changes which is a Good Thing. Less to document and more flexibility on the server end. But it wouldn’t protect us from changes in the rel attributes. In the RESTbucks world it doesn’t help us if you suddenly have to respond to the question “Would you like syrup with that?” before processing or if during payment processing a 3dsecure step is added. It’s just about moving the contract between client and server to a different place. It’s probably a good thing because we probably do change URLs a lot, or would if we could do it without breaking clients. But I can’t see how it could ever be the magical solution to end out of band communication.

I think what HATEOAS does is shift the point at which APIs break and changes the boundary point of the client server contract. Yes you can tweak your server URLs by adding an extra layer of defence. But you can’t have clients that don’t know about the actions they might expect to take when interacting with a service and the data they might be expected to fill in.

If you go to Starbucks then there’s some prior implicit knowledge about interacting with the overall service. The staff can guide you through the transaction. But if you’re a tribesman from Papua New Guinea just landed in the UK as part of a documentary, then you could be lacking a lot of prior information that’s necessary for a smooth transaction. You’re going to need to understand some basic capitalism, what’s being traded, why you might want a Hazelnut Latte, and to be honest if you speak English, it’s gonna be a lot easier. I think HATEOAS is the member of staff. They can’t teach you English. They can’t give you free coffee. But if you have some level of implicit understanding of the overall process they can guide you through to the next steps.

So what’s the upshot of this?

HATEOAS seems like it will give you a slightly more friendly contract between clients and servers. When writing our API documentation (because I can’t see a way around the pre-determined information) we can write thinner more human readable documents. We can say things like:

These are the services currently available:

• sign_in
• read_messages
• tweet

To see what fields you need to fill in you can visit the relevant URLs from the API root.

And yes you could make this straightforward XHTML and then a person can even go and browse in a web browser. And then your API interaction could becomes the same as a human, filling in forms and clicking links (Capybara interaction). I think that’s probably writing more code than you need to, but it does give us an interesting way of exploring an API. e.g. as a developer I’d rather write

twitter.sign_in user: "mark@example.com", password: "password"
twitter.tweet "HATEOAS could make my life easier"

#than

visit "/sign_in"
fill_in "user", with: "password"
fill_in "password", with: "password"
click_button "Sign in"
fill_in "tweet", with: "HATEOAS could make my life easier"
click_button "Tweet"

Discoverable APIs could lead to some interesting ways that we’ll be able to make our code and APIs more maintainable and easier to developer in the future. I like the idea of an API that’s also browsable. And I think as API documentation it’s a killer idea. In this way we reduce prior information, in that someone writing a client still goes and looks at the API in advance, but you don’t have external documentation explaining the API that you need to keep in sync.

It won’t get us around the fact that if you change your fields or validations or the order in which it’s possible to progress through a service/application, you will break your clients.

Your clients are still tied to the specific version of the API they were written against but they’ll be a lot easier to update if you have a web browsable version of your API and discoverable URLs.

OK this has nothing to do with Vim, but sometimes I want to talk about just Ruby too.

I came across the following issue:

This is OK:

"abcアイウ".scan(/\p{Katakana}/)
=> ["ア", "イ", "ウ"]

This isn’t:

"abc".scan(/\p{Katakana}/)
invalid character property name {Katakana}: /\p{Katakana}/

Workaround works:
line = "abc"
"#{line}☃".encode('utf-8')[0..-2].scan(/\p{Katakana}/)

=> []

So set your String instance variables that you want to use like that:

”#{line}☃”.encode(‘utf-8’)[0..-2]

Or wrap your accessor up like that and you should be good to go without the rather unexpected behaviour of monster car 鬼車 oniguruma throwing a wobbly just because it doesn’t find a match.

I have a setting to automatically trim whitespace at the end of lines on save.

I found that the pre-commit gem was still warning me about trailing whitespace

Turns out that the file was in DOS mode and was saving ^M style line endings and pre-commit didn’t like it.

No amount of whitespace trimming would save the day. I ended up viewing the file in hex mode before I noticed. Anyway, if you run into this issue, just do the following and re-save the file.

:set ff=unix

Do you like the relative line numbers setting?

set relativenumber 

I do, but sometimes I’d like to see the actual line numbers when going to a line in a stack trace for example.

Anyway, try: 

let g:show_relative_numbers = 1



function! ToggleRelativeLineNumber()

  if g:show_relative_numbers

    set number!

  else

    set relativenumber!

  endif

  let g:show_relative_numbers = !g:show_relative_numbers

endfunction


nnoremap ;l :call ToggleRelativeLineNumber()<CR>

Then you can just do:

;l

to swap between relative and normal line numbers

I use Chrome, Terminal and MacVim now on a daily basis.

One thing that absolutely kills me is inconsistency in user interfaces.

When I switch between tabs in any application I expect it to be pretty damn predictable.

Save yourself hours and hours in a year by going to your System Preferences > Keyboard > Application Shortcuts 

And work your magic to make it look like this:

A google search at present returns questions about closing all buffers easily in vim but not straightforward tutorials - which always makes me feel that the issue is not being properly solved. Well it took me a while to wade through the initial search results and actually find something that closes all saved files.

(Next post will deal with a destructive close all including unsaved files.)

Hopefully here’s an easy start for you to what should be a really basic use-case in my opinion.

Close all buffers saving everything (shutting down vim)

:wqall! 

Close all buffers without saving (shutting down vim)

:qall!

Close all buffers visually but leaving them open

:only 

Probably none of these are what you wanted.

What about actually closing them?

Why isn’t this easy? Not sure. But here’s what I’m doing for now:

Actually close all (saved) buffers with

F4

Actually close all (saved) buffers except this one with

F3

Here’s how:

map <F3> :BufOnly<CR> "Close all other buffers but this one
map <F4> :BufOnly<CR>:bd <cr> "Close all buffers

Requirements

Install the BufOnly plugin

https://github.com/vim-scripts/BufOnly.vim

You are using Pathogen and Git submodules aren’t you?

Issues

There seems to be a problem with this approach not updating the tabs at the top of GVim on Ubuntu so I’ll look at that too for the next post.