JeffProd | Blog

Web scraping by watching requests

Fri, 22 Oct 2021 01:38:39 +0200

Web scraping is a technique used to extract content from websites in order to archive data in a structured way. The data is extracted directly from the source code. The work consists in searching it in a jungle of HTML tags.

It is however much easier to retrieve this data in a structured format. This is possible by intercepting web requests between your browser and the website.

This method works, among others, on social networks or classifieds websites (but make sure to respect their terms of use).

How to clone a PC and boot it into a virtual machine

Sun, 20 Jun 2021 09:36:58 +0200

Backing up a complete computer and booting it into a virtual machine is possible with just a few Linux commands. The tools are free and open source.

Create your own i3/sway status bar with some bash and python

Sun, 21 Jun 2020 11:19:31 +0200

i3 is a tiling window manager on Linux. It supports tiling, stacking, and tabbing layouts, which it handles dynamically. The advantage is that you no longer waste time organizing your windows with the mouse. With i3 the layout is automatic. The windows open by occupying the available space. Using customizable keyboard shortcuts, you can move them around, display them full screen, arrange them horizontally or vertically. This is much more convenient and faster than using the mouse.

Svelte JS : VSCode configuration and internationalization

Wed, 06 May 2020 10:04:05 +0200

Svelte is a set of tools to develop Javascript applications (yet another new JS framework). It is a direct competitor of React or VueJS. After having tried it, I confirm that this project is to be followed with interest.

Automating tasks in a virtualized OS

Thu, 05 Dec 2019 11:06:16 +0100

Or how to perform actions in a virtual machine without leaving the terminal…

From Linux, I was looking for a way to automate the compilation of a software on Windows, and without using Wine which is not an official Microsoft product.

VirtualBox is a multi-platform virtualization software. It comes with VBoxManage, a command line tool for managing virtual machines.

Web scraping with Electron

Sat, 09 Mar 2019 08:12:35 +0100

Web scraping is a technique for extracting content from websites in order to archive data in a structured way. Be careful, however, to respect the terms of use of the website concerned.

Electron is a framework for creating native Windows/Mac/Linux applications with web technologies (Javascript, HTML, CSS). It includes the browser Chromium, fully configurable.

Is there a better way to code a portable application with a graphical user interface to scrape a given site ?

Rate limiting API requests

Sat, 22 Dec 2018 14:25:59 +0100

You have released a public API and you want to avoid abuse ? It is possible to limit access to a specific number of requests within periods of time. For example, if a user sends two successive requests when you only allow one request every 5 seconds, he will get the following answer :

Rate limit exceeded, please try again in 4 seconds.

Build your own Hugo website search engine

Wed, 05 Sep 2018 07:36:39 +0200

The goal is to build a static site search :

without installing any JS module on your computer
without indexing stop words, i.e. words like “and, or, with, the…”
indexing words once by page (no duplicates)
your next created pages will be added automatically, without any additional work.

Solving a Rubik's pocket cube with a graph database

Fri, 17 Mar 2017 19:14:01 +0200

A pocket cube can have 3 674 160 different positions. For a “No-SQL” database, it’s a piece of cake.

Neo4j is a graph database. In this tool, each node could be a cube position, and eges from one node to another could be a move.