RSS (Atom) and Fediverse

2022-03-29 22:00 | Amelia Meyer | The Moody Blues--"Steppin' In A Slide Zone" | frustrated

So I thought, "Hey, you know what would be neat? If I could post my new entries from the blog and from my fictional works on Fediverse! I see a bunch of people doing that, so surely it must be easy!"

Yeah.

About that.

Stumbling block the first

I have 2FA enabled on my account like anyone ought to on anything that offers it. mastodon.py does not support 2FA for its automagical app authorization dance, and does not document the format for the magical config files that it produces. It only took like two hours reading the Python code of two projects to figure this out! Neat!

For future reference, this is what it wants:

Client credentials file:

<client_id>
<client_secret>
<instance_uri>

User credentials file:

<access_token>
<instance_uri>

Stumbling block the second

Why is this important?

Well, turns out basically all the search results for variations on the theme of rss feed to mastodon are projects built on mastodon.py. Makes sense, you use that for the Mastodon side, and something like feedparser for the feed side, and plug it all together. Easy!

Except, with no 2FA support, all of them choke immediately on trying to auth themselves.

So what you have to do is go to the Developer section in your profile and create a new app by hand, then create the two files listed above.

Now things like feediverse or feed2toot will auth! Yay!

Stumbling block the third

So, cool, now we have the Mastodon side working. Neat. How about them RSS?

Well, turns out feediverse has very limited ability to customize what your toots will look like, so I skipped that one quickly.

feed2toot looks great! Reasonable formatting options (though not anything with newlines, so...), named feeds, etc.

Cool. So I get it authed and configured and do a dry run:

(.venv) agmlego@brigid /opt/feed2toot $ feed2toot --debug --dry-run -c feed2toot.ini
configured stdout level 10
document declared as us-ascii, but parsed as utf-8
document declared as us-ascii, but parsed as utf-8
No feed could be retrieved. Leaving.

Whelp. That is certainly..."useful" output. Turns out what that is telling me is that feedparser is in picky mode and producing no entries for feeds with "bozo exceptions". I got there by, again, digging through several projects' source code found by several variations of searches for the document encoding error.

Eventually, I determined that I could get feedparser to give me more information by using it directly in the Python REPL:

>>> import feedparser
>>> feed = feedparser.parse('https://agmlego.com/categories/cat_blog.xml')
>>> feed['bozo_exception']
CharacterEncodingOverride('document declared as us-ascii, but parsed as utf-8')

Huh, OK. So I check the feed source:

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="../assets/xml/rss.xsl" media="all"?>
<rss version="2.0"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:atom="http://www.w3.org/2005/Atom">

Sure as shit looks like UTF-8 to me...

Well, yeah, but apparently nginx defaults to content-type: text/xml for XML files, and Atom feeds really ought to be application/atom+xml or similar.

Fine. I edit the vhost config and add a location stanza which grabs the top-level and category feeds from Nikola, as well as some other feeds for other things I serve:

location ~ (feeds|categories|rss).*\.xml$ {
    ## override content-type ##
    types { } default_type "application/atom+xml; charset=utf-8";

    ## override header (more like send custom header using nginx) #
    add_header x-robots-tag "noindex, follow";
}

Then I run it again:

(.venv) agmlego@brigid /opt/feed2toot $ feed2toot --debug --dry-run -c feed2toot.ini
configured stdout level 10
lockfile /opt/feed2toot/.lock created.
found feed entry https://agmlego.com/posts/fiction/flight/
Would toot with visibility "unlisted": A new fictional work at https://agmlego.com/posts/fiction/flight/ -- Flight!: "Tower information papa. Automated weather observation one one five three Zulu. Winds one five knots at two five zero degree. Visibility one zero. Sky conditions few at seven thousand. Temperature two one, dewpoint one five. Altimeter three zero one three. Approach runway two seven left in use on #artemis #lucille #janais #gamma #moira #tlmc
found feed entry https://agmlego.com/posts/fiction/breakfast-at-the-den/
Would toot with visibility "unlisted": A new fictional work at https://agmlego.com/posts/fiction/breakfast-at-the-den/ -- Breakfast at The Den: Janais padded sleepily into the kitchen, failing to stifle a yawn. She plonks her forehead into Kian's mid back, wrapping her arms around the woman's bare midriff.
"Hey cutie, breakfast is almost ready. Is the blue-eyed one up yet?" Kian rumbled, deftly turning several eggs over in her pan #artemis #janais #moira #kian #tlmc
found feed entry https://agmlego.com/posts/fiction/beginnings/
Would toot with visibility "unlisted": A new fictional work at https://agmlego.com/posts/fiction/beginnings/ -- Beginnings: I close my eyes and lean back into the dingy couch, my fingers interlaced behind my head, stretching up and back. This place is a good place, a home away from home.
The pungent smell of laser-ablated leather mixes with notes of cutting fluid and oak sawdust. Over the powerful hum of the ve #lavernge #glaive #janais #moira #kian #tlmc
found feed entry https://agmlego.com/posts/fiction/deal-with-it/
Would toot with visibility "unlisted": A new fictional work at https://agmlego.com/posts/fiction/deal-with-it/ -- Deal With It: Moira sat in a private meeting room at a coworking space, anxiously watching for her friend to arrive. She idly fidgets with a black and silver pronoun pin on her blouse. Before her on the table, a single sheet of paper. A stylized logo of a single-edged polearm underlined a company name in the r #moira #ashe #tlmc
found feed entry https://agmlego.com/posts/blog/hashtag-first-post/
Would toot with visibility "unlisted": A new blog post at https://agmlego.com/posts/blog/hashtag-first-post/ -- Hashtag First Post: So, hey, welcome, settle down, take a seat. This is my blog. It is little, but might eventually not be little.
It was fun putting it together; this blog runs on nikola, which is super easy to use and work with.
However, and this is a fun story for
found feed entry https://agmlego.com/posts/blog/hands-held-high-15/
Would toot with visibility "unlisted": A new blog post at https://agmlego.com/posts/blog/hands-held-high-15/ -- "Hands Held High", A Reflection After 15 Years: Fuck. I just. I was winding down, right? New OT, new Marcel Vos.
Then blammo. This absolutely impeccable live performance of Linkin Park's "Hand #feelings #music #pain #war
found feed entry https://agmlego.com/posts/blog/writing-is-good-actually/
Would toot with visibility "unlisted": A new blog post at https://agmlego.com/posts/blog/writing-is-good-actually/ -- Writing is Good, Actually: So I just published the first three vignettes in the fictional universe that has been living rent-free in my head for the last five or so years.
My polycule a few weeks ago goaded me into writing some of it down, and, uh, suddenly I find I #creativity #fiction #writing
Removed lock file.

Hot damn.

Stumbling block the fourth

Well, so the above is actually after an hour of futzing with the slicing and toot_max_len fields in the config. Despite my home instance supporting toots of up to 1024 characters, I could not convince feed2toot to make toots any longer than around 800 characters, and even then, it was stripping off the tags for anything longer than around 500, no matter what I set otherwise.

So I guess we stop here. Good enough, I am worn down and tired from battling this all evening.

Software was not written to be used.

Concluding thoughts

You know how to do this on Twitter?

Have an account on IFTT or automation.io or dlvr.it or circleboom or zapier
Give the service your RSS feed link
Do the oauth dance with usually at most one click in the service
Done