Andy Sylvester andysylvester

News Archive - Project Spec

Background

News websites publish new stories daily, but older content quickly becomes hard to find — buried behind pagination, missing from site search, or removed entirely. This project solves that by continuously collecting articles from RSS feeds and making them searchable through a simple web interface, creating a persistent archive that grows over time.

Target users: Anyone who wants to search or browse archived news stories across multiple sources.

Concept

News Archive - Project Spec

Background

News sites/newspapers create stories on a daily basis and make them available to readers. The stories are updated daily, but finding older content is difficult. How can this problem be solved?

Concept

An integrated RSS collection/search engine

	#!/usr/bin/env python3
	"""
	sitemap_to_rss.py
	=================
	Convert a sitemap XML file (URL or local path) into an RSS 2.0 feed.

	Supported namespaces
	--------------------
	• http://www.sitemaps.org/schemas/sitemap/0.9 (standard sitemap)
	• http://www.google.com/schemas/sitemap-news/0.9 (Google News extension)

	Listening at http://ron.mystatustool.com:2004
	[Object: null prototype] {
	session: 's3750',
	domain: 'retiredkalam.com',
	port: '80',
	path: '/?rsscloud=notify',
	feed: 'https://retiredkalam.com/feed/' }
	s3750: POST https://retiredkalam.com/?rsscloud=notify https://retiredkalam.com/feed/
	{ method: 'POST',
	uri: 'https://retiredkalam.com/?rsscloud=notify',

	{
	"events": [
	{
	"method": "GET",
	"path": "/feedupdated",
	"params": {
	"url": "https://rsscloud4.wordpress.com/feed/",
	"challenge": "0f43e414a6ab0416c14bc1a0b73ce2"
	},
	"myResponse": "0f43e414a6ab0416c14bc1a0b73ce2",

	const myVersion = "0.4.0", myProductName = "clouddemo";

	const fs = require ("fs");
	const utils = require ("daveutils");
	const qs = require ("querystring");
	const request = require ("request");
	const davehttp = require ("davehttp");
	const xml2js = require ("xml2js");
	const reallysimple = require ("reallysimple");

	{
	"events": [
	{
	"method": "GET",
	"path": "/feedupdated",
	"params": {
	"url": "https://rsscloud.andysylvester.com/feed/",
	"challenge": "f03b10e367c0384fe2e5ad4e075ab1"
	},
	"myResponse": "f03b10e367c0384fe2e5ad4e075ab1",

	root@AndyDO-03:~/rssDave# node clouddemo.js
	davehttp.startup: launching on port == 443, v0.4.37.

	davehttp.startup: config == {
	"port": 443,
	"flPostEnabled": true,
	"flLogToConsole": true,
	"flTraceOnError": false,
	"defaultFeedUrl": "https://rsscloud.andysylvester.com/feed/",
	"thisServer": {