support subscribing to home page microformats2 h-feed and h-entrys

[tantek]

Modern sites are publishing feeds on their home pages marked up with the h-feed and h-entry microformats with higher fidelity (more entries, more content, sooner, more reliable) than legacy XML formats (RSS/Atom).

http://microformats.org/wiki/h-feed
http://microformats.org/wiki/h-entry

It would be great if feedbin supported directly subscribing to a people's home pages, and only fallback to legacy feed files if there's no h-feed or h-entry on the home page.

There are microformats2 parsers:
http://microformats.org/wiki/microformats2#Parsers

Including an open source Ruby microformats2 parser:
https://github.com/G5/microformats2

That feedbin could use to do this.

Thanks for your consideration!

[benubois]

Hi @tantek,

Thanks for the suggestion! I like the idea of microformats.

I tried out a few things and my initial impression is that this might not be a great fit. At least not right now.

I recognize the chicken and egg nature of standards, but at a minimum I would need to rely heavily on existing parser implementations as writing one from scratch would expand the scope greatly and likely be over my head :)

I tried out the ruby parser you mentioned, but did not have great luck with it.

I was able to find two examples of h-feed from the wiki and ran into a couple of blockers. Please let me know if I'm missing something or if there are other example sites I should try out.

irb(main):001:0> require "microformats2"
=> true
irb(main):002:0> collection = Microformats2.parse('http://www.sandeep.io/')
irb(main):003:0> collection.entries.count
NoMethodError: undefined method `entries' for #<Microformats2::Collection:0x007fcc81e2bbe8>
  from (irb):3
  from /Users/ben/.rbenv/versions/2.1.4/bin/irb:11:in `<main>'

irb(main):004:0> collection = Microformats2.parse('http://tantek.com/')
irb(main):005:0> collection.entries.count
NoMethodError: undefined method `entries' for #<Microformats2::Collection:0x007fcc83083e58>
    from (irb):5
    from /Users/ben/.rbenv/versions/2.1.4/bin/irb:11:in `<main>'

Based on the tests I would expect entries to be available.

I also tried the node implementation. This one had better results but I was not left with a great impression. Using http://www.sandeep.io/ as a test URL I noticed a couple of format issues and potentially missing data. I'm not sure if these are problems with the site or the parser.

{
  "items": [{
    "type": ["h-feed"],
    "properties": {
      "name": ["sandeep.io"],
      "author": [{
        "value": "Sandeep Shetty Generalist. Punk Programmer. Founder, Simpthings.",
        "type": ["h-card"],
        "properties": {
          "url": ["http://www.sandeep.io/"],
          "uid": ["http://www.sandeep.io/"],
          "photo": ["http://www.gravatar.com/avatar/64a4d7587e6e3db5b5ff8152b25eb1bc?s=420&d=mm&r=g"],
          "name": ["Sandeep Shetty"],
          "note": ["Generalist. Punk Programmer. Founder, Simpthings."]
        }
      }],
      "category": ["#indieweb", "#converspace", "#webmention", "#technology", "#rssb", "#thinkingoutloud", "#ubuntu", "#design", "#rss", "#facebook", "#email", "#css", "#microformats", "#comments", "#activitystreams", "#php", "#research", "#readlater", "#tagging", "#irc", "#markdown", "#simplicity", "#atom", "#blogroll", "#cli", "#ux", "#dellxps13", "#pumpio", "#colors", "#composer", "#api", "#fitness", "#github", "#web", "#travel", "#bookmarks", "#wordpress", "#socialobjects", "#time", "#dunbar", "#mumbai", "#twitter", "#privacy", "#deconditioning", "#skeleton", "#mobile", "#idno", "#bootstrap", "#xkcd", "#pgp", "#copy", "#books", "#writing", "#json", "#anonymity", "#training", "#keybase", "#exercise", "#ssh", "#real-time", "#fun", "#otxt", "#openid", "#pubsubhubbub", "#PNL", "#persona", "#feed", "#webfinger", "#religion", "#webappdev", "#philosophy", "#rfc", "#url-design-hack", "#ftp", "#http", "#shopify", "#viralheadlines", "#phpish", "#statemachines", "#yaml", "#apt", "#parsers", "#git", "#whatsapp", "#crossfit", "#comicstrip", "#flu", "#bodyweighttraining", "#pingback", "#https", "#ssl", "#markaround", "#webdesign", "#jsync", "#protocol", "#toml", "#trackback", "#sick", "#slow-web", "#permalinks", "#funtheory", "#animation", "#java", "#poem", "#mysql", "#pagination", "#less", "#photography", "#craftsmanship", "#health", "#syntax", "#javascript", "#immunity", "#art", "#SEO", "#g11n", "#i18n", "#L10n", "#webdesigner", "#37signals", "#jobsite", "#recruitment", "#resume", "#LinkedIn", "#drugs", "#markup", "#trailing-tags", "#music", "#mipslinux", "#machinetags", "#ui", "#rdf", "#graph", "#h-reader", "#url", "#hypertext", "#googleplus", "#myspace", "#rest", "#taproot", "#salary", "#foundation", "#videos", "#verb", "#plaintext", "#responsive", "#todo", "#bootstrapanonymous", "#usecases", "#webaction", "#grid", "#python", "#activitydialog", "#websockets"]
    },
    "children": [{
      "value": "Break out of ssh after it locks Every time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it's place but it annoyed me because it didn't feel right. Today, I found a way to kill the session and regain access to the terminal window by using the following key comnination: Enter + ~ + . Another nifty thing I learned is the ability to suspend a ssh session with ~^Z and then unsuspend it with fg+Enter. References: https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/ https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/ http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/ http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/ http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key #ssh #cli #ubuntu # Break out of ssh after it locks Every time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it's place but it annoyed me because it didn't feel right. Today, I found a way to kill the session and regain access to the terminal window by using the following key comnination: `Enter` + `~` + `.` Another nifty thing I learned is the ability to suspend a ssh session with `~^Z` and then unsuspend it with `fg`+`Enter`. References: * <https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/> * <https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/> * <http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/> * <http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/> * <http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks> * <http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key> #ssh #cli #ubuntu 7 Jan 2015 0 Comments 0 Likes 1 Reposts 0 Mentions",
      "type": ["h-entry"],
      "properties": {
        "category": ["ssh", "cli", "ubuntu"],
        "name": ["# Break out of ssh after it locks Every time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it's place but it annoyed me because it didn't feel right. Today, I found a way to kill the session and regain access to the terminal window by using the following key comnination: `Enter` + `~` + `.` Another nifty thing I learned is the ability to suspend a ssh session with `~^Z` and then unsuspend it with `fg`+`Enter`. References: * <https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/> * <https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/> * <http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/> * <http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/> * <http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks> * <http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key> #ssh #cli #ubuntu"],
        "summary": ["# Break out of ssh after it locks Every time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it's place but it annoyed me because it didn't feel right. Today, I found a way to kill the session and regain access to the terminal window by using the following key comnination: `Enter` + `~` + `.` Another nifty thing I learned is the ability to suspend a ssh session with `~^Z` and then unsuspend it with `fg`+`Enter`. References: * <https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/> * <https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/> * <http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/> * <http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/> * <http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks> * <http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key> #ssh #cli #ubuntu"],
        "content": [{
          "value": "# Break out of ssh after it locks Every time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it's place but it annoyed me because it didn't feel right. Today, I found a way to kill the session and regain access to the terminal window by using the following key comnination: `Enter` + `~` + `.` Another nifty thing I learned is the ability to suspend a ssh session with `~^Z` and then unsuspend it with `fg`+`Enter`. References: * <https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/> * <https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/> * <http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/> * <http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/> * <http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks> * <http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key> #ssh #cli #ubuntu",
          "html": "# Break out of ssh after it locks\r\nEvery time I suspend my laptop (by closing the lid) I inevitably have at least one terminal with an ssh session open to a remote host that ends up freezing that terminal on unsuspend. I usually just close that terminal window/tab and start a new one in it&#039;s place but it annoyed me because it didn&#039;t feel right.\n\nToday, I found a way to kill the session and regain access to the terminal window by using the following key comnination: `Enter` + `~` + `.` \n\nAnother nifty thing I learned is the ability to suspend a ssh session with `~^Z` and then unsuspend it with `fg`+`Enter`.\n\nReferences:\n\n* &lt;https://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-kill-dead-ssh-sessions/&gt;\n* &lt;https://www.void.gr/kargig/blog/2006/01/24/ssh-escape-characters/&gt;\n* &lt;http://www.techrepublic.com/blog/it-security/how-to-escape-ssh-sessions-without-ending-them/&gt;\n* &lt;http://grack.com/blog/2011/02/23/ssh-escape-sequences-or-dont-kill-9-that-process/&gt;\n* &lt;http://askubuntu.com/questions/29942/how-can-i-break-out-of-ssh-when-it-locks&gt;\n* &lt;http://serverfault.com/questions/293360/escape-sequence-for-the-escape-key&gt;\n\n\n#ssh #cli #ubuntu"
        }],
        "url": ["http://www.sandeep.io/192"],
        "published": ["2015-01-07T20:10:24+0000"]
      }
    }]
  }],
}

• The entry is missing a title
• The html version actually seems to have all HTML stripped

[karadaisy]

Thanks for checking it out @benubois! I am not a Rubyist, but I have heard that the Ruby parser is a bit different from the Node, Python, and PHP implementations. I'll report this problem to them, sounds like there's either a bug in the parser or the docs...

The results you got with the node parser are actually correct -- the "name" field corresponds to title, so for short notes (like tweet-length, untitled posts), it will often be the same as the content. And in this one particular case, Sandeep takes the "unorthodox" approach of including unparsed Markdown as the content of his post entries. On pretty much every other site that publishes h-feed, content -> html would give you regular old HTML.

My site (https://kylewm.com) or @tantek's will give you the more expected results.

Big fan of Feedbin, btw, keep up the good work!

[dissolve]

@benubuis, the Ruby parser has since been completely rewritten and greatly improved, wonder if we could look in to this again

[dissolve]

Oops @benubois

[pstuifzand]

@benubois I saw this post and tried to find out how to parse the h-feed of sandeep. In Irb.

> require "microformats2"
> collection = Microformats2.parse('http://www.sandeep.io/')
> collection.items[0].type                # => h-feed
> collection.items[0].children(:all)      # => array of h-entry
> collection.items[0].children(:all)[0].name              # => h-entry title
> collection.items[0].children(:all)[0].url               # => h-entry url 
> collection.items[0].children(:all)[0].content[:html]    # => h-entry html content

In microformats you have multiple items in each of the properties, so everything is an array. The microformats parser hides that by having multiple ways of getting the info:

name     => is first item of array
name(0) => also
name(1) => second
name(:all) => array of all 'name'

I hope this sheds some light. Would also be great to support microformats.