User-Agent and Referer Headers [Universal Feed Parser]

User-Agent and Referer Headers

Universal Feed Parser sends a default User-Agent string when it requests a feed from a web server.

The default User-Agent string looks like this:

UniversalFeedParser/3.0 +http://feedparser.org/

If you are embedding Universal Feed Parser in a larger application, you should change the User-Agent to your application name and URL.

Example: Customizing the User-Agent

>>> import feedparser
>>> d = feedparser.parse('http://feedparser.org/docs/examples/atom.xml', \
...         agent='MyApp/1.0 +http://example.com/')

You can also set the User-Agent once, globally, and then call the parse function normally.

Example: Customizing the User-Agent permanently

>>> import feedparser
>>> feedparser.USER_AGENT = "MyApp/1.0 +http://example.com/"
>>> d = feedparser.parse('http://feedparser.org/docs/examples/atom.xml')

Universal Feed Parser also lets you set the referrer when you download a feed from a web server. This is discouraged, because it is a violation of RFC 2616. The default behavior is to send a blank referrer, and you should never need to override this.

Example: Customizing the referrer

>>> import feedparser
>>> d = feedparser.parse('http://feedparser.org/docs/examples/atom.xml', \
...     referrer='http://example.com/')

← ETag and Last-Modified Headers

HTTP Redirects →