Over the past year I've been slowly working on improvements to the Feedsearch API that I published in late 2017, and wrote about in my previous post.

The biggest difference is that the old Feedsearch library has now been replaced with the new Feedsearch Crawler library, which is available for use in other projects as a PyPI package. The crawler is now able to make HTTP requests asyncronously, drastically speeding up the number of URLs that can be checked for feeds in the same amount of time, which in turn increases the feed detection rate in cases where the feed URL is not properly advertised.

Read more…

I've just published a new microsite and API called Feedsearch, which provides an easy way to search websites for RSS, Atom, and JSON feeds.

Feedsearch is primarily meant to be a thin public API wrapper around the Feedsearch Python package, which I wrote to support feed search capabilities in my side project Auctorial. I couldn't find a public library or service that both searched for feeds and returned feed and site metadata, so I wrote this one. In addition to the API, the site allows users to search for feeds using a standard webform.

Read more…

Update: Airbnb's Customer Support just got back to me by email, and confirmed that Prepaid phones are fine for Verification, it's just that for some reason my current number won't verify from their end either, and they don't know why. That explanation makes sense to me, weird bugs and edge cases happen with code and complex systems like phone networks. Makes a lot more sense than the idea that they just wouldn't support Prepaid phones like their first response in their Twitter DM claimed. So all up, a couple points off their score for the completely wrong first answer, but many more points gained for the response and explanation within two and a half hours by the support specialist, and for the discount voucher. Good job.

Original post: I just tried verifying my Swiss phone number with Airbnb, and they won't let me. According to their support people on Twitter, Airbnb doesn't support Prepaid phone numbers for verification purposes.

This is news to me, because I've had two different Prepaid numbers verified in the past, from two different countries. Both of which required no personal identification to buy the Prepaid card, unlike here in Switzerland, where even Prepaid cards require filling out a form and showing proof of address. That means that there's no technical block against Prepaid numbers, and no real enforced legal case against them either, or they wouldn't have accepted my other numbers.

Read more…

Recently I noticed that an application I'm hosting on AWS Elastic Beanstalk wasn't logging the Client IP Address of my users. After a lot of digging I found the issue was to do with the configuration of the Nginx servers running on the EC2 instances and in my application's Docker container, and in the TCP passthrough configuration of the Elastic Load Balancer.

So here's a quick post about my particular setup and the configuration I used to fix this issue of Client IP Addresses not being forwarded.

Read more…

Express 4.0 was recently released, with one of the major changes being that connect middleware is no longer available on the express module. This means no basic http authentication unless you do it yourself. Other people may find this easy, but I had to look it up, so to help people like me here are some quick instructions on getting it going again.

Read more…

It generally goes without saying these days that testing is an important part of the development process. However, testing is not always the easiest thing to master, and it can be difficult to know where to start, especially when it comes to learning a new language. Node.js adds an extra twist on this difficulty, in that your code is running Asynchronously, and you will often be dealing with HTTP calls.

Having just been through the process of learning Node.js and working on a simple application that deals with calling and receiving responses from an external API, the most painful thing I found was that there aren't many tutorials on how to test HTTP requests and responses that I find particularly clear. So, this tutorial aims to provide clear information on how to effectively test your requests and responses, with a little bit about testing emitted events.

Read more…

Bittorrent Sync has been getting a lot of press lately for being a really good file synchronisation solution. I was thinking about installing it on my systems today, to act as a secure, decentralised alternative to Dropbox. Then I had a sudden thought: "Wait, is Bittorrent Sync open source?".

Turns out no, it's not.

Well, crap.

Read more…

This post aims to be a fairly thorough guide to getting SSL up and running on an Nginx server, largely for my own reference in future, but it may prove useful to anyone else who stumbles across it. I'm going to assume that you have a working installation of Nginx, or know how to get one going. If not, then bookmark this page and find a tutorial on how to get it going first, such as the section on Nginx in my post here, or the Installation and Beginner's Guide pages on the official site.

Read more…

Having reinstalled Ubuntu a few times on various machines, I figured it's about time that I recorded a few of the post installation steps I take, so that I don't have to keep looking them up. The instructions below are for Ubuntu 13.10, and can be done in any order.

Read more…

Over the past few days I have been reading Understanding Power: The Indispensable Chomsky, and even though I've previously read about a few of the cases he describes, the sheer scale of the duplicity Chomsky presents here is still shocking. If even one of his narratives about the complicity of the Media in covering up or spinning atrocities is true, then it ought to rock people's beliefs about the way political power and the Media operate. I agree with Aaron Swartz's view, that it really does change the way you look at the world.

Read more…