Improving blogger experience
Almost forgot to mention how I improved blogger experience.
I needed a TOC (Table of contents), such that a reader can quickly jump to the section he's interested in.
Blogger does NOT offer this out of the gate.
So I dug, dug deeper and I found 2 solutions which actually create a TOC. But neither was good enough.
One only fetched only level of <H> tags, the other one fetched all but got lost as soon as there's more complicated HTML code between the <H> tags.
So I used some parts of both and improved upon it: You paste this in the HTML BEFORE everything:
<div id="myToc">
</div>
<hr />
<div id="myContents">
And you paste this AFTER everything
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script type="text/javascript">
countChapters = $("#myContents > h1, #myContents > h2").length
chapters = $("#myContents > h1, #myContents > h2")
flagLevel = false
var toc = ''
for (i = 0; i < countChapters; i++) {
chapter = chapters[i]
chapterTitle = chapters[i].textContent;
chapters[i].setAttribute("id", "chapter" + i);
if ('H2' == chapter.tagName) {
if (flagLevel) {
toc += "<li><a href='#chapter" + i + "'>" + chapterTitle + "</a></li>";
} else {
toc += "<ul><li><a href='#chapter" + i + "'>" + chapterTitle + "</a></li>";
}
flagLevel = true
} else {
if (flagLevel) {
toc += "</ul>"
flagLevel = false
}
toc += "<li><a href='#chapter" + i + "'>" + chapterTitle + "</a></li>";
}
}
document.getElementById("myToc").innerHTML = toc;
</script>
Then it will parse all H1/H2 tags and create a TOC at the top.
By fine-tuning the flagLevel thing you could even have it parse H3 tags.
Who's interested: let me know and I enhance the code for you!
This technique can actually be used on every HTML page
Install web server
This is an easy one on macOS, thanks to this
I had home-brew (brew) already installed. So I just needed to follow a few steps.
Apple delivers a built-in apache version, yet the default configuration is a bit not so straight forward. With the approach from above I had the web server running in a few minutes, pointing to my directories.
Make (bot) results accessible from afar
Here I needed to think/play a little bit. Years ago there were DynDNS services, for free. Nowadays they charge for their service. What they basically do is provide a hostname which is mapped to your dynamic public IP from your home router.
So I tried to build my own "rough" DynDNS solution.
I started with https://www.whatismyip.com and https://github.com/cheeriojs/cheerio a tool which lets you parse HTML on the server (and not on the client side). Here's a nice tutorial https://dev.to/diass_le/tutorial-web-scraping-with-nodejs-and-cheerio-2jbh.
On the way I learned that jQuery is a thing of the past and one should use React or Vue now, so I did a 30 minutes primer into Vue. Nice, but couldn't understand why/where it's better than jQuery. With Vue you have to "wrap" your tags into "v-" tags in order for it to work. Works well for applications, but it's a pain if you want to do very small things, like things I did with my visual tool to analyse charts.
Anyway: I got cheerio to read the contents of whatismyip.com and it took me almost 2 hours to figure, that they would not return my IP, as they realise the request is not from a user, but from a machine.
But there's other tools, like this one: http://ip-api.com/json Very straight forward. Returns my IP and that's all I need.
Now what to do with my IP? Store it on google drive? Do they have an API? After a few minutes tinkering I figured: I will use iCloud. Create a document there with a link to my "own" web server. And that's what I did.
Please don't judge me on this code: it's a prototype and does what it needs to do:
module.paths.unshift('/Users/michael/NodeJS');
const axios = require("axios").default;
url = 'http://ip-api.com/json'
const fetchHtml = async url => {
try {
const { data } = await axios.get(url);
return data;
} catch {
console.error(
`ERROR: An error occurred while trying to fetch the URL: ${url}`
);
}
};
async function doIt() {
var result = await fetchHtml('http://ip-api.com/json')
var path = '/Users/michael/Library/Mobile Documents/com~apple~CloudDocs/HTML/'
var ip = result.query
var fs = require('fs');
fs.writeFileSync(path + 'myServer.html', `</br></br></br></br></br><H1><a href="http://${ip}/heartBeats">myServer</a></H1>`)
fs.writeFileSync(path + 'myTool.html', `</br></br></br></br></br><H1><a href="http://${ip}:3000">myTool</a></H1>`)
}
doIt()
Important here is the first line. This is required as cron, which will run the script, does not know where my node modules are installed
If fetches the IP address from my router (via this URL), stores it in a link in a HTML file in a folder in iCloud where I can access it from anywhere in the world.
I only needed to to some port forwarding in my router. Port 80/3000 need to forward to my machine.
Interestingly, once I set it up, he, the router software, told me: my Mac will from now on have a static address. How nice!
Final piece: I created a a soft link in my web documents folder to the folder where I store my bot heart beats. So now I can call my server like:
http://my.ip.address/heartBeats
and it will show me the directory listing of my heartBeats so I can check from everywhere if my bots are running.