Skip to content

Instantly share code, notes, and snippets.

View mtrunkat's full-sized avatar
💻
Hiding behind the keyboard

Marek Trunkát mtrunkat

💻
Hiding behind the keyboard
52 followers · 2 following
View GitHub Profile
@mtrunkat
mtrunkat / Dockerfile
Last active July 26, 2021 09:45 — forked from jancurn/Dockerfile
WARNING for future me: Do not ever delete this thing! It's used in integration tests.
# Here you choose the base Docker image for the actor. Apify provides the following images:
# apify/actor-node-basic
# apify/actor-node-chrome
# apify/actor-node-puppeteer
# However, you can use any other image from Docker Hub.
# For more information, see https://apify.com/docs/actor#base-images
FROM apify/actor-node-basic
# Copy all files and directories from the directory to the Docker image
COPY . ./
@mtrunkat
mtrunkat / server.js
Last active March 5, 2020 12:03
Blogpost: https://blog.apify.com/live-view-of-running-actors-on-public-hostname-18a88546b0f2
const express = require('express');
const { APIFY_CONTAINER_PORT, APIFY_CONTAINER_URL } = process.env;
const app = express()
app.get('/', (req, res) => {
res.send('Hello World!');
});
app.listen(APIFY_CONTAINER_PORT, () => {
@mtrunkat
mtrunkat / main.js
Created April 5, 2018 13:21
Hacker News crawler using Apify SDK (PuppeteerCrawler and RequestQueue classes)
const Apify = require('apify');
Apify.main(async () => {
// Get queue and enqueue first url.
const requestQueue = await Apify.openRequestQueue();
const enqueue = async url => requestQueue.addRequest(new Apify.Request({ url }));
await enqueue('https://news.ycombinator.com/');
// Create crawler.
const crawler = new Apify.PuppeteerCrawler({
@mtrunkat
mtrunkat / page-function.js
Last active March 5, 2020 12:05
Blogpost: https://blog.apify.com/how-to-never-miss-a-beat-on-ever-changing-websites-8cccfa25f5cb
function pageFunction(context) {
var $ = context.jQuery
var posts = $('.athing').toArray(); // All posts as array of DOM elements
var $moreLink = $('.morelink'); // Link to next page
// If crawler is scraping 2nd, 3rd, ... page then
// context.request.referrer.pageFunctionResult contains
// result from previous pages.
var prevResult = context.request.referrer
? context.request.referrer.pageFunctionResult
@mtrunkat
mtrunkat / output.json
Last active March 5, 2020 12:05
Blogpost: https://blog.apify.com/how-to-never-miss-a-beat-on-ever-changing-websites-8cccfa25f5cb
[{
"https://github.com/apas/athena": 100,
"https://github.com/achillesrasquinha/bulbea": 99,
"https://keratin.tech": 98,
"https://github.com/dhruvio/js_express-parser-combinators": 97,
"https://www.pagedash.com/": 96,
"https://github.com/monostream/tifig": 95,
"https://javalin.io/news/javalin-1.0.0-stable.html": 94,
"http://www.goodoldweb.com/": 93,
"http://vuemc.io": 92,
@mtrunkat
mtrunkat / daemon.sh
Last active June 12, 2017 20:05
Run bash script forever with CRON with concurrency 1
#!/bin/bash
# Original: http://bencane.com/2015/09/22/preventing-duplicate-cron-job-executions/
#
# This script executes ./cmd_loop.sh and save process ID (PID) in file ./forever.pid.
# Everytime it's executed it checks for PID in ./forever.pid and if process is still
# running then exists with nonzero code. Otherwise it executes ./cmd_loop.sh.
PIDFILE=./forever.pid
@mtrunkat
mtrunkat / openssl-to-pem.sh
Last active March 5, 2020 12:04
Conversion pf p12 certificate to public and private pem key
# Get private key
openssl pkcs12 -in yourP12File.pfx -nocerts -out privateKey.pem
# Get public key
openssl pkcs12 -in yourP12File.pfx -clcerts -nokeys -out publicCert.pem
@mtrunkat
mtrunkat / docker-mongo-repair
Last active March 19, 2024 06:28
Run "mongo --repair" in Docker container that cannot start because of MongoDB error
#!/bin/bash
# See https://github.com/docker-library/mongo/pull/63
docker run --rm --volumes-from my-mongo-server mongo unlink "/data/db/mongod.lock"
docker run --rm --volumes-from my-mongo-server mongo --repair
@mtrunkat
mtrunkat / docker-host-ip.sh
Created January 9, 2017 19:23
Get IP address of host machine from docker container
#!/bin/bash
/sbin/ip route|awk '/default/ { print $3 }'
@mtrunkat
mtrunkat / waitforstring.sh
Last active December 27, 2016 13:55
Bash function that waits for string to appear in file.
#!/bin/bash
# Waits for string $2 to appear in the file $1.
# Meanwhile it is printing output of the file $1.
# If err or ERR string is find in the file $1 then exits with error code 1.
# Example use:
# waitforstring /var/log/myapp.log "MongoDB started"
function waitforstring {
local LAST_LINE=1
local READ_LOG