358 questions
0
votes
1
answer
50
views
Feedparser errors during SSl read operation when accessing NASDAQ RSS Feeds
By utliizing Python 3.12, Feedparser 6.0.11, ca-certificates installed
When attempting to read this RSS feed: https://www.nasdaq.com/feed/rssoutbound?category=Financial+Advisors. Feedparser library ...
- 2,465
0
votes
1
answer
47
views
How to add rows to dataframe in pandas
Building a scraper for news:
import feedparser
import pandas as pd
from datetime import datetime
archive = pd.read_csv("national_news_scrape.csv")
pd.set_option('display.max_colwidth', None)...
- 7,710
0
votes
1
answer
30
views
Not sure why URL isn't being passed into a scraper of RSS feeds
Just want to scrape news feeds from RSS.
import feedparser
import pandas as pd
from datetime import datetime
archive = pd.read_csv("national_news_scrape.csv")
# Your list of feeds
feeds = [...
- 7,710
-1
votes
1
answer
154
views
Correctly sort RSS items by time
I'm getting RSS items from different RSS channels. And I'd like to sort them correctly by time and take into account the time zone, from the latests to the oldests. So far, I have the following code:
...
- 4,190
0
votes
0
answers
29
views
Python Feedparser Not Showing Entries
I've watched a lot of videos about feedparser library in tutorials everybody use this library like in my code but my code is not working... Anyone help me? my code
I still can't get output even though ...
- 1
0
votes
1
answer
129
views
How to scrape data into PythonAnywhere database
I have a database on PythonAnywhere and credentials in place.
The aim is to scrape a whole load of news websites and chuck the data into a new website using Flask.
Here's my code for a section of my ...
- 7,710
0
votes
1
answer
2k
views
ModuleNotFoundError: No module named 'feedparser'
I have Python 3.11 and I have done everything expect Anaconda thing in this video below to try and fix this issue.
https://www.youtube.com/watch?v=RvbUqf3Tb1s&t=181s&ab_channel=TechWithTim
...
- 1
0
votes
1
answer
142
views
Formatting a tweet from python using tweepy
I'm trying to tweet my movie reviews from Letterboxd using python's tweepy but I can't format the tweet. I tried printing the tweet without the square brackets or printing the title and link to the ...
- 3
0
votes
1
answer
87
views
Python rss feed gets AttributeError
Running this code:
import feedparser
Feed = feedparser.parse('http://www.reddit.com/r/python/.rss')
pointer = Feed.entries[1]
print (pointer.summary)
print (pointer.link)
I get this error:
...
- 165
2
votes
0
answers
224
views
How do I update RSS feed with feedparser in discord.py?
I'm working on a custom bot to send a message in a Discord channel every time a new post is made on a subreddit. With this, it just sends a link to the same post every 30 seconds, the post never ...
- 21
0
votes
2
answers
1k
views
How to scrape news articles from cnbc with keyword "Green hydrogen"?
I am trying to scrap news article listed in this url, all article are in span.Card-title. But this gives blank output. Is there any to resolve this?
from bs4 import BeautifulSoup as soup
import ...
11
votes
1
answer
12k
views
Skipping analyzing "feedparser.util": module is installed, but missing library stubs or py.typed marker
How do I fix this error? It seems feedparser does not support mypy typings? I could not find a typeshed implementation for feedparser
UPDATE 1
I see an option called ignore_missing_imports that I can ...
- 6,208
2
votes
0
answers
232
views
Feedparser pubdate gives error when trying to store into a table with asyncpg
I am trying to store feedparser data into a Postgres database via asyncpg and I am getting an error while storing pubdate that is of type timestamptz in the database
I have a Postgres table test with ...
- 6,208
1
vote
1
answer
2k
views
Using Python to get Substack posts without scraping
I want to create a dataframe of Substack posts from all the newsletter I subscribe to. But using feedparser + Substack's RSS feeds only seem to go back ~20 posts—even if a particular newsletter has ...
- 968
-1
votes
1
answer
51
views
Why is my pythonscript breaking after hours of working fine?
I wrote a small simple program which checks the RSS feed of a german news site, saves the pubDate, title, description and link of the last article and saves it to a txt file. After running over 11 ...
-1
votes
1
answer
104
views
Feedparser not returning values, only metadata
I'm using feedparser to get info from a public database (https://knesset.gov.il/Odata/ParliamentInfo.svc/KNS_Bill()).
Each of my entries looks as follows
When accessing specific properties:
url = '...
- 1
0
votes
2
answers
54
views
How to cycle through indices
so in this script I am writing to learn python, I would like to just put a wildcard instead of rewriting this whole block just to change line 2. what would be the most efficient way to consolidate ...
2
votes
1
answer
3k
views
How to use an rss feed in python?
I have never worked with an RSS feed before, I can't seem to find the url of the feed.
The page which is offering the RSS Feed:
https://www.sec.gov/edgar/browse/?CIK=717826&owner=exclude
I am ...
- 4,035
0
votes
0
answers
198
views
Python feedparser's bozo is of bool type instead of int
Disclaimer first: I am a python newbie.
I am using feedparser version 6.0.8 (checked using pip freeze | grep feedparser) to parse Twitter feeds from random Nitter instances (using twiiit.com). Python ...
- 803
1
vote
1
answer
92
views
How to create Article tags from splitting the Title in Django
I want to create articles tags from the title of a RSS feed post. Then save the tags into a DB with a post_id of the title i got the tags from at the same time. Something like this:
Title = "...
- 153
1
vote
1
answer
226
views
how to handle 30x when using feedparser to parse rss url
Now I am using Python 3 feedparser to parse some RSS url, this is my code:
if __name__ == "__main__":
try:
feed = feedparser.parse("https://ucw.moe/feed/rss")
...
- 37.9k
1
vote
1
answer
730
views
Cannot get thumbnail in rss feed in laravel
I am using willvincent feed reader to parse rss feeds, But i cannot seem to get the thumbail of the images,
Here is my code
Route::get('feed', function(Request $request) {
$f = FeedsFacade::make('...
- 73
2
votes
4
answers
303
views
Accessing duplicate feed tags using feedparser
I'm trying to parse this feed: https://feeds.podcastmirror.com/dudesanddadspodcast
The channel section has two entries for podcast:person
<podcast:person role="host" img="https://...
- 8,714
0
votes
0
answers
104
views
what's the use of feed(input()) in Python?
from HTMLParser import HTMLParser
class MyHTMLParser(HTMLParser):
def handle_starttag(self, tag, attrs):
print ('Start :', tag)
for ele in attrs:
print ('->', ele[0]...
- 1
0
votes
1
answer
175
views
Feedparser not parsing email body properly
I'm trying to parse the below (which came from a gmail email body) using feedparser by feeding it in as a raw text feed. I'll be displaying it on a page so I want it to look like it did in the email ...
- 179
1
vote
0
answers
123
views
How can I serve a modified version of an RSS feed?
The problem: I use inoreader.com to monitor a set of RSS feeds. Each of the feeds is generated by pubmed.gov and contains academic journal articles. But there is an annoying quirk where pubmed encodes ...
- 4,667
1
vote
1
answer
706
views
AttributeError: object has no attribute 'published' when parsing CNN source
I'm facing this problem since parsing CNN.RSS site. It only get first 7 entries then i get this problem. Below it my log .. Please help me :(. Thanks you
This is my code:
import feedparser
url = &...
- 31
0
votes
1
answer
530
views
python feedparser and getting multiple categories (category) within a item
import feedparser
import webbrowser
feed = feedparser.parse("https://seekingalpha.com/feed.xml")
feed_entries = feed.entries
for entry in feed.entries:
article_title = entry.title
...
2
votes
1
answer
497
views
Error using cx_freeze with a feedparser import: ModuleNotFoundError: No module named 'sgmllib'
I am trying to use cx_freeze to freeze an app that uses Feedparser. The app works perfectly if I launch it from the command line using python. However, when I try to freeze it using cx_freeze, I don't ...
- 409
0
votes
1
answer
243
views
How to add a link to text in tkinter text widget with feedparser?
I would like to build a simple rss reader. I use the following code:
import tkinter as tk
import feedparser
import sqlite3
import webbrowser
NewsFeed = feedparser.parse("https://www.wired.com/...
- 129
0
votes
1
answer
444
views
Import RSS with FeedParser and Get Both Posts and General Information to Single Pandas DataFrame
I am working on as a python novice on an exercise to practice importing data in python. Eventually I want to analyze data from different podcasts (infos on the podcasts itself and every episode) by ...
- 31
1
vote
1
answer
383
views
Retrieving JSON Title, URL from Pygooglenews feedParser
I'm trying to retrieve values from pyGoogleNews (https://github.com/kotartemiy/pygooglenews)
that is using feedParser to output the values.
Code Snippet as follows:
from pygooglenews import ...
- 2,218
1
vote
2
answers
103
views
how can I clean my code content once bs4 scrapes the code snippet?
I am trying to scrape all the data content inside code , but however my code looks like kinda weirdy on code_snippet = soup.find('code') since it display different data as the following:
<code ...
3
votes
1
answer
411
views
Extract value from a list formatted like a JSON file in Python
I have a function that returns an image URL from an RSS feed (I'm using feedparser). The problem is that it returns a strange formatted list.
My feed has a key called media_content. The code used to ...
- 125
3
votes
2
answers
4k
views
How to query arXiv for a specific year?
I'm using the code shown below in order to retrieve papers from arXiv. I want to retrieve papers that have words "machine" and "learning" in the title. The number of papers is ...
- 2,978
0
votes
0
answers
299
views
Feedparser is not sanitizing html text downloaded via asyncio aiohttp
I am downloading feed using aiohttp asyncio
Feedparser is supposed to sanitize input text with its _HTMLSanitizer class to accept only specific tags
The sanitizer does not work, any suggestions?
...
- 6,208
0
votes
2
answers
592
views
How do I pass raw untrusted text to feedparser.parse method in Python?
I am trying to use feedparser to parse text which I download using asyncio aiohttp library
The feed text is available HERE (Large document, hence not pasting here)
The documentation of feedparser....
- 6,208
1
vote
1
answer
218
views
Feedparser, no etag and modified attributes
How could I know if the feed has changed since last check if none of these attributes exists for a given feed ?
Example with this one from a nintendo news website:
d=feedparser.parse('https://www....
- 274
0
votes
2
answers
2k
views
How to update RSS Feed every 5 seconds in Python using Flask
I did a lot of research and nothing relevant worked. Basically I am trying to scrape RSS Feed and populate the data in a table format on a webpage created using Python Flask. I have scraped the data ...
- 21
1
vote
1
answer
187
views
Output of feedparser in python unexpectedly truncated
I'm writing a piece of code that parses information from an RSS feed. I am storing the parsed informations for later research. In the case at hand I'd like to store info such as [Name, Surname, Type ...
- 11
1
vote
0
answers
43
views
What represents this more exactly
I'm building a RSS-Feed parser to extract news from various websites. One thing tho that makes me confused is:
<html lang="nl" dir="ltr" prefix="content: http://purl.org/rss/1.0/modules/content/ ...
0
votes
0
answers
296
views
Python Feedparser pubdate to one timezone
I need to parse RSS feed. I am using feedparser on python. Basically my task is to run script every N seconds and check for last feed. I come up with idea to check is date is younger than 15 seconds ...
- 187
0
votes
1
answer
270
views
Parsing <cap:event>, <cap:urgency> tags value using feedparser in python from a Atom/xml feed
I am using feedparser for parsing data from an Atom/xml feed file. The link for the file is:
https://alerts.weather.gov/cap/oh.php?x=0
This is a feed that is put out by the National Weather Service ...
- 31
0
votes
3
answers
61
views
Printing a list in Python3
I am using feedparser to pull some data from a news channel RSS page.
There are around 54 title in the page.
However I want to print for example 10 details.
I tried over and over but i couldn't do ...
- 119
0
votes
1
answer
212
views
Issue while using python rss feed with feedparser
I am trying to get the data from a rss feed using python 3.7.7 and feedparser.
I am able to get the simple information like feed['title'] but I am unable to get feed['ht:approx_traffic'] which is one ...
- 1,495
0
votes
1
answer
112
views
memory leak on FeedPaser 5.2.1?
Recently i get to parse a RSS feed using FeedPaser 5.2.1, accidently i find a continuous memory increase as my app. running without break. is there any mistake made by me? Any help would be highly ...
- 35
0
votes
0
answers
249
views
List index out of range error only when run on server - python
I hope you can help me with my issue
I am currently coding a bot on discord with python using discord.py
There is a portion of my code which, for the time being, is used to retrieve the latest (or ...
- 21
1
vote
1
answer
1k
views
How to detect with feedparser if there are new items in an RSS channel?
I have the following code. When you understand the code, you can look at the two comments with
the capital letters. I could test if there are new items in the channel with insert or ignore but
I'm ...
- 4,190
2
votes
1
answer
2k
views
How to use feedparser with proxy in python 3?
I'd like to use a proxy and feedparser. This is how I'd do in urllib2:
import urllib2, feedparser
proxy = urllib2.ProxyHandler({"http":"proxy:port"})
d = feedparser.parse('http://www.shop.inonit.in/...
- 3,388
0
votes
1
answer
469
views
Sorting a large list of rss feed items by publish_parsed or updated_parsed
I have a large list of rss feed items that I'm attempting to sort by key 'published_parsed' like so:
allheadlines.sort(key=lambda hl:hl[0]['published_parsed'], reverse=True)
The problem is this is ...
- 179