Això és Artesans Digitals, un lloc web al qual trobaràs informació, guies i altres publicacions relacionades amb la sobirania digital, programari lliure, com fer una millor gestió dels dispositius electrònics, trucs per millorar la seguretat en línia, i altres temes relacionats.

Parlo de com aprofitar dispositius per a tenir els teus propis serveis i dades, i no haver de pagar per "el cloud". Creu-me, no cal utilitzar Google Drive.

Dedicat a tots aquells artesans digitals, curiosos i amants de la tecnologia, que vulguin alliberar-se de les cadenes a les que ens intenten lligar.

Totalment obert, gratuït, i en Català. Per què algú ho havia de fer.

Si estàs cansat de que els anuncis et persegueixin, de no saber què fer d'aquell portàtil antic que tens tirat per casa, i tens una mínima curiositat per a saber com protegir les teves dades davant dels gegants tecnològics i el futur distòpicament artificial al que ens estem apropant... Benvingut, aquí t'hi trobaràs com a casa.

Tot el que trobareu aquí són recopilacions d'informació que jo també he trobat de forma gratuïta per internet. Simplement resumida i organitzada per a que sigui més fàcil d'utilitzar.
Si algú us intenta cobrar per aquest material, segurament treballi a una Big4.

Nota: Els posts previs al juny de 2024 han sigut importats d'altres plataformes on els havia compartit anteriorment.

Per un Internet que ens faci més lliures, i no més oprimits, passeu, apreneu i compartiu el que creieu més important amb el vostre cercle proper. Potser demà sereu vosaltres qui els ajudareu a alliberar-se.

Tens poc espai al Google Drive?

Les fotos del mòvil t'ocupen molt però no les vols penjar a iCloud per si les hackegen?

Tagradaria tenir un calendari amb el qual Google no pugui accedir a tota la teva agenda personal?

Voldries enviar missatges privats i segurs a tots aquells amb qui comparteixes documents?

Tant si et sents identificada amb totes les preguntes anteriors, com només amb alguna, avui et porto una solució sigui quina sigui la teva situació.

Amb el post d'avui, podràs deixar de fer servir les solucions tancades dels gegants tecnològics, i seguir disfrutant dels serveis essencials pel teu dia a dia de forma totalment personal i privada:

• Calendari
• Contactes
• Documents
• Fotos
• Notes
• Xat personal i grupal
• i molt més…

I tot això, només amb un sol programa que podem instal·lar de forma facilíssima al nostre servidor.

Aquest programa és Nextcloud, i a la guia d'avui l'instal·larem al servidor Linux que prèviament hem configurat a un dispositiu allotjat a la nostra xarxa wifi de casa, i us ensenyaré com el podeu fer servir de forma personal, o també oferir-lo com a alternativa a Google Drive, calendari personal, etc. per als vostres amics / familia / comunitat, o qui vulgueu.

La seguretat i la usabilitat van de la mà, només cal posar una mica més d'esforç del que ens han acostumat les grans corporacions intrusives

Disclaimer: Aquesta guía NO està pensada per a una instal·lació “professional” de Nextcloud per a empreses. Si voleu tenir una instància per a la vostre empresa, recomano instal·lar Nextcloud-AIO en comptes de la forma com ho farem en aquesta guia.

Disclaimer 2: Pròximament penjare un article sobre com instal·lar Nextcloud per a una empresa, els passos són una mica més extensos que a la guia d'avui degut a l'increment de seguretat que requereix gestionar una empresa.

La principal diferència entre aquesta instal·lació de Nextcloud i una de "professional", és que aquesta NO la obrirem a internet. Per tant, només qui estigui connectat a la VPN (recordeu que en l'article sobre com instal·lar el nostre servidor "núvol" personal hem instal·lat també Tailscale) o a la xarxa wifi de casa podrà entrar a Nextcloud.

Això ens proporciona alguns inconvenients, com limitar el número d'usuaris que hi accedeixen, però a canvi ens dóna una major seguretat per a tota la informació personal que hi pengem, ja que ningú hi podrà accedir des d'internet directament.

Mans a la obra.

Si preferiu una versió en anglès, aquesta guía està basada en la següent: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-nextcloud-on-ubuntu-20-04

Instal·lació

Instal·lar Nextcloud al nostre servidor és molt fàcil. Només ens hi hem de connectar per SSH i executar les següents comandes:

sudo apt update
sudo apt upgrade
sudo apt install snapd
sudo snap install nextcloud

Un cop executades, ja tenim nextcloud instal·lat. Si ho volem comprovar executem sudo snap info nextcloud, on veurem tota la informació corresponent a la instal·lació (paquets addicionals, base de dades, versió...).

Fixeu-vos que no hem hagut d'escollir res de res. Aquesta instal·lació es fa per defecte al port 80 (HTTP), per tant quan accedim a la IP del nostre servidor, serem redirigits automàticament a Nextcloud.

Nota: Oju! Si ja teniu algun altre programa executant-se al port 80, com APACHE o NGINX. És possible que us doni problemes en aquest cas. Si no és estrictament necessari, recomano desinstal·lar NGINX o canviar-li la configuració predeterminada.

Ara a través del nostre ordinador personal, entrem al navegador i escrivim la IP del servidor. Podem utilitzar tant la de la xarxa wifi de casa (serà semblant a 192.168.1.xxx) o la de Tailscale.

El primer que veurem serà que ens demana crear un compte d'administrador. El creem i guardem les credencials segures. Aquest usuari serà el que haurem d'utilitzar en cas de tenir algun problema. Més endavant crearem el nostre usuari personal.

Seguidament ens permet escollir quines apps volem instal·lar. Si teniu suficient espai al servidor, podeu instal·lar totes les recomanades i més endavant ja eliminareu les que no utilitzeu.

Finalment hauriem de poder veure una pantalla similar a la següent:

Felicitats! Ja tenim el nostre Nextcloud operatiu! Fàcil oi?

Però abans d'utilitzar-lo, recomano crear un certificat SSL auto-firmat. El teu navegador possiblement encara detecti que la connexió es insegura, però almenys les comunicacions amb el servidor sempre aniran xifrades.

Per fer-ho, des de la terminal del servidor executem les següents comandes:

sudo ufw allow 80.443/tcp
sudo nextcloud.enable-https self-signed

Ens hauria d'apareixer un output similar al següent:

OutputGenerating key and self-signed certificate... done
Restarting apache... done

Ara si recarreguem la pàgina al navegador, hauria de detectar el certificat. Encara que ens diu que no és segur, procedim per què sabem que el certificat l'hem signat nosaltres.

Si això us passa amb alguna altre pàgina web externa, no hi entreu o almenys assegureu-vos de no afegir-hi informació personal, ja que aquesta pot ser interceptada.

Troubleshooting

Si teniu problemes per accedir tot i haver seguit tots els passos anteriors, és possible que hagueu d'afegir la IP de Tailscale, o fins i tot la vostre IP local com a "trusted domain". Si heu afegit un nom de host per accedir-hi, també l'haureu d'afegir al fitxer indicat.

Configuració i usuaris

Seguim ara amb la creació del nostre usuari personal. Anem a la cantonada superior dreta, i fem click a "Usuarios".

Creem un nou usuari personal i si ho creieu oportú li doneu permisos d'administrador.

En cas d'oferir aquest Nextcloud a amics / familiars / comunitat, també haureu crear els seus usuaris, ja que no hem configurat cap servidor de correu per a que els notifiqui. Haureu d'afegir una contrassenya per a ells, però els hi haureu de fer saber que se la han de canviar un cop hi entrin. En cas que la perdin des de l'usuari administrador els hi podràs resetejar.

I fins aquí seria la configuració bàsica inicial.

Ara ja només us queda trastejar una mica el funcionament de Nextcloud per dins, mirar si voleu instal·lar alguna App addicional des de la seva botiga d'apps interna (totes són gratuïtes), i pujar els primers documents / fotos / contactes, o el que vulgueu.

Sincronització WebDAV i Apps Mòbils

Un dels grans potencials de Nextcloud, és que podem sincronitzar una carpeta del nostre ordinador directament amb el servidor.

Per tant, no ens farà falta entrar cada cop al navegador per a guardar fitxers, sinó que fins i tot si estem sense connexió, podem estar treballant sobre un document, guardar-lo a la carpeta sincronitzada amb Nextcloud, i un cop tornem a tenir connexió, aquest es penjarà automàticament al nostre servidor.

Per a crear aquesta carpeta sincronitzada, primer necessitarem descarregar la Aplicació de Nextcloud per escriptori (disponible per a Windows, MacOs i Linux. També per a Android i iOS): https://nextcloud.com/es/install/#install-clients

Un cop descarregada, la obrim i hauriem de veure alguna cosa similar a la següent imatge:

Fem click a "Iniciar Sesión en tu Nextcloud" i afegim la IP de Tailscale del nostre servidor. Important afegir la de Tailscale, i no la local, ja que sinó quan no estiguem connectats al wifi de casa nostre no ens podrem connectar a Nextcloud.

Si ens diu que el certificat no és segur, li diem que hi confiem i continuem.

Ara ja només ens farà falta escollir un nom per la carpeta que volem afegir al nostre ordinador, i que estigui sincronitzada amb el servidor. No us feu la vida difícil i poseu-li un nom fàcil... com "Nextcloud" 😉.

I... voilà! Ja tenim una carpeta sincronitzada amb el nostre servidor! Podeu comprovar que tenim els mateixos documents si accedim des de la carpeta com des del navegador:

Ja queda poc, ara farem el mateix però pels nostres dispositius mòbils.

A través de la App Store que creieu més oportuna (si és Android podeu utilitzar F-Droid o Aurora Store), descarregueu-vos les següents apps:

• Nextcloud --> Per als documents
• Nextcloud Talk --> Per als xats
• Nextcloud Notes --> Per a les notes

Un cop descarregades, haureu de seguir el mateix procediment de connectar-vos a la IP del servidor. Òbviament necessiteu tenir Tailscale instal·lat també al vostre mòbil.

Amb tot l'anterior, ja podeu accedir des de qualsevol dels vostres dispositius als documents, xats i notes personals sense haver d'utilitzar cap aplicació amb la que no confiem.

Cada cop som més aprop de ser soberans amb les nostres dades, i no permetem que s'utilitzin en contra nostre i/o per benefici d'altri.

Heu de ser conscients que aquest pas us fa a vosaltres les principals responsables d'aquestes dades, i per tant heu de planificar bé les còpies de seguretat de les més importants. Poc a poc anireu guanyant confiança en la gestió d'aquestes, i us adonareu que és molt més fàcil ser nosaltres els propis responsables del que les grans corporacions ens han fet creure.

L'internet és un lloc obert, i simplement hem de triar on i com volem utilitzar-lo. No fa falta acceptar directament tots els termes i condicions que ens posin a davant, només per què "és la opció fàcil" (creu-me, en realitat no ho és).

Opcional: Sincronitzant el Calendari i Contactes del mòbil

Nextcloud ens ofereix també la opció de sincronitzar els nostres contactes i el calendari del nostre mòbil, però s'han de seguir uns passos extres, no és tant automàtic com amb les apps anteriors.

Així com per sincronitzar la carpeta al nostre ordinador hem utilitzat una tecnologia anomenada WebDAV, pel calendari farem servir CalDAV i pels contactes CardDAV.

Aquests són protocols oberts que podem utilitzar amb altres aplicacions de codi obert que no fa falta que siguin les pròpies de Nextcloud (de fet, Nextcloud no en té cap de pròpia).

Un cop instal·lats, podreu tenir el calendari del mòbil i el del vostre ordinador sincronitzats amb totes les vostres reunions i esdeveniments, rebre notificacions, afegir recordatoris, etc. Igual que amb qualsevol altre app de Calendari, pero en comptes de donar-li la vostra agenda personal directament a l'empresa que hi hagi darrere la app, aquestes dades estaran emmagatzemades al vostre servidor.

I pels contactes, "más de lo mismo". Quan afegiu un nou contacte, podreu emmagatzemar-lo a Nextcloud, en comptes de al vostre compte de Google, o iOS. Així si més endavant us canvieu el mòbil, podreu recuperar el contacte pero sense que els gegants tecnològics puguin crear grafs de connexions amb els teus contactes.

Bé, prou de xerrar. Mans a la obra.

Les apps que fan falta instal·lar al mòbil són:

• DAV5x - Sincronització CardDAV i CalDAV
• Etar - Calendari de codi obertç

Primer entrem a Dav5x per a crear la connexió pertinent amb el nostre servidor. Si ja heu seguit les seccions anteriors, aquí serà el mateix. Afegir la IP de Tailscale del vostre servidor, i iniciar sessió a Nextcloud si us ho demana. Escolliu sincronitzar tant els contactes com el calendari (CardDAV i CalDAV).

Un cop sincronitzat, pels contactes podeu importar si ho voleu els que tingueu guardats a altres llocs, i així us feu una còpia a Nextcloud dels contactes actuals.

Pel calendari, escollim tots els calendaris que vulguem sincronitzar. Ens hauria de detectar automàticament si tenim més d'un perfil de calendari, com "Personal", "Feina", "Dies festius", etc.

Quan ja estiguin tots sincronitzats, ja podeu entrar a l'altre aplicació que hem instal·lat, Etar.

A dins, ens dirigim a la configuració, i escollim que el volem connectar a CalDAV. Això ens redirigirà a la app Dav5x i haurem d'escollir quins són els calendaris que volem connectar a Etar. Si no hi ha cap error inesperat, haurieu de veure tots els vostres esdeveniments del calendari també a Etar.

Actualitzant a la última versió

Si veieu que Nextcloud s'ha d'actualitzar, ho podeu fer de forma molt fàcil amb la següent comanda des de la terminal del servidor:

sudo snap refresh nextcloud

Notes finals

Benvingut a la teva nova pila d'apps sobirana. Tenir un Google pixel amb GrapheneOS no és suficient (tot i que molt recomanable), la tecnologia sobirana no ha de ser avorrida o inutilitzable. La seguretat i la usabilitat van de la mà, només cal posar una mica més d'esforç del que ens han acostumat les grans corporacions intrusives i, a poc a poc però amb bona lletra, aniràs guanyant confiança i aconseguint tornar a ser un individu soberà.

la tecnologia sobirana no ha de ser avorrida o inutilitzable
- Markush

En cas de tenir qualsevol dubte, trobaràs les meves dades de contacte a la pàgina de contacte.

Si aquest blog t'ha aportat valor o has après alguna cosa nova, pots ajudar-me compartint-lo a xarxes socials, o deixant-me una donació a través de la icona del llamp que trobareu a la part inferior dreta.
Sí, aquí només s'accepta Bitcoin ⚡

En la actual era digital, on estem evolucionant cap a un futur distòpic, l'auto allotjament s'ha de convertir en la norma... Tornem a les arrels, per entendre per què cauen les fulles.

Benvinguts al primer blog sobre com tenir el teu propi servidor i diverses aplicacions auto allotjades per a deixar de dependre de les grans corporacions en la gestió de les nostres dades.

En aquesta guia, explicaré com crear el teu propi servidor "cloud" des de zero. Sense utilitzar cap sistema operatiu preinstal·lat, posarem un sistema operatiu Ubuntu Server nou al maquinari que trieu i el configurareu perquè tingui un dispositiu segur i llest per actuar com a servidor personal.

A mesura que hi afegeixi programari nou, seguiré creant i penjant més guies per explicar com he instal·lat i estic utilitzant la resta de programari. Estigueu atents si voleu aprendre a convertir-vos en un ninja d'Internet.

Nota: Tot el que trobareu aquí (fins i tot la pàgina web que estàs veient), està auto allotjat a un servidor que tinc a casa, i ho faig servir al meu dia a dia. El cost total que he invertit en aquest equipament no supera els 250 euros, incloent el mateix servidor, i perifèrics. Si hagués de pagar per a tots els serveis que hi tinc instal·lats, en menys d'un any ja hauria pagat més del que m'ha costat la inversió inicial.
I el valor real és tot el coneixement que guanyes pel camí.
Si hi ha qualsevol cosa que no us quedi clara, no dubteu en contactar-me.

Problema

Depenem en gran mesura de serveis de tercers, tant per a les nostres dades com per als nostres diners. Estem constantment sota vigilància per les nostres accions i no podem ser lliures fins que no aprenguem a utilitzar els serveis digitals de manera privada.

La constant cessió de les nostres dades permet que les grans corporacions facin negocis amb elles, i han de gastar cada cop més materials, recursos i energia en centres de dades per gestionar-les.

Solució

Instal·lar serveis i aplicacions en un ordinador allotjat a casa. Deixem de regalar les nostres dades per al benefici d'altri i aconseguim ser realment lliure al món digital, sense perdre cap avantatge en utilitzar Internet amb els nostres propis dispositius electrònics.

Doneu sentit als dispositius que, en cas contrari, acabarien a la brossa, contribuint a la reducció de residus i fomentant la reutilització de materials informàtics que acabarien obsolets. L'auto allotjament de serveis i l'ús de programes de codi obert ens beneficien tant a nosaltres com al nostre planeta.

La revolució comença amb petits gestos. Si entenem el poder que tenim i l'usem, mai més ens el podran treure.

Converteix-te en un individu sobirà.

Instal·lació

0. Preparacions

Abans de començar a embrutar-nos les mans (o el teclat), és important que tinguem tot el material necessari per construir el nostre propi servidor cloud personal. En cas contrari, haureu de córrer a la vostra tenda electrònica de confiança a comprar l'adaptador més estrany que se us pugui ocórrer perquè el vostre servidor no té un port determinat.

Assegureu-vos de saber quins ports podeu utilitzar amb el vostre maquinari nou per connectar-hi les següents peces:

• Un portàtil antic o mini-pc (NUC, raspberry, etc.)
• Monitor
• Teclat
• Cable Ethernet
• Emmagatzematge extern (HDD o SDD) de com a mínim 1TB
• Un pendrive

Un cop ho tenim tot, podem començar a preparar la nostra nova configuració.

Abans de connectar-lo a la corrent, connecteu el servidor al router mitjançant un cable Ethernet, connecteu l'emmagatzematge extern a través d'USB o SATA i connecteu tant el teclat a un port USB com la pantalla amb un cable HDMI o DisplayPort.

També, abans d'iniciar el servidor, prepararem el nou sistema operatiu que alimentarà la vostra màquina. En aquest cas, utilitzarem Ubuntu Server 22.04. Podeu trobar l'enllaç de descàrrega aquí: https://ubuntu.com/download/server

Descarregueu la versió més recent i emmagatzmeu-la localment al vostre ordinador.

Descarregueu Balena Etcher: https://etcher.balena.io/ → aquest programari us serà útil cada vegada que vulgueu flashejar un sistema operatiu a un dels vostres dispositius.

Finalment, connecteu el vostre pendrive a l'ordinador, obriu Balena Etcher i trieu el fitxer que heu acabat de descarregar (fitxer .iso d'Ubuntu Server) per flashejar-lo al vostre pendrive.

Un cop finalitzat el flashejat, podeu extreure el pendrive i estarem llestos per iniciar la instal·lació real del vostre servidor personal.

1. Instal·lació del Sistema Operatiu

En aquesta secció explicarem el procés d'instal·lació pas a pas, incloent-hi instruccions especials per utilitzar tot el disc, però també un disc extern i crear una estructura de dades per muntar-lo (/mnt).

ABANS d'encendre el servidor per primera vegada, connecta-hi la memòria USB que prèviament has flashejat amb Ubuntu Server.

Ara, podem engegar el servidor per primera vegada. Connecta'l a la corrent i prem el botó d'engegada, si n'hi ha.

Habitualment, en el primer inici es mostrarà automàticament una finestra amb diverses opcions. En aquest moment, has de prémer una de les tecles que s'haurien d'indicar per accedir al menú d'arrencada o simplement moure't per la llista d'opcions disponibles amb les fletxes del teclat i seleccionar l'opció d'arrencada del menú.

Allà veuràs l'opció d'arrencar des del dispositiu extraïble (memòria USB). Fes clic en aquesta opció. Ara començarà el procés de flashejar tot el sistema operatiu nou al teu servidor cloud personal.

Utilitza les tecles FLETXA AMUNT, FLETXA ABAIX i ENTER del teclat per navegar per les opcions. Segueix les instruccions següents:

1. 1. A la primera pantalla, selecciona l'idioma que prefereixes (es recomana anglès)
2. 2. Si hi ha disponible una actualització del programa d'instal·lació, selecciona "Actualitzar al nou instal·lador", prem ENTER i espera
3. 3. Selecciona la distribució del teclat i la variant (es recomana espanyol per a usuaris nadius de castellà. Si us hi sentiu còmodes també podeu escollir anglès) i prem [done].
4. 4. Segueix seleccionant "Ubuntu Server" com a base per a la instal·lació, fins a [done], i prem ENTER.
5. 5. Selecciona la connexió de xarxa d'interfície que vols utilitzar (es recomana Ethernet) i anota la teva IP obtinguda automàticament mitjançant DHCP. (Normalment 192.168.x.xx). Prem [done] Tingues en compte que més tard assignarem una IP estàtica al servidor.
6. 6. Deixa l'opció següent buida si no vols utilitzar un proxy HTTP per accedir-hi. Prem [done].
7. 7. Si no vols utilitzar un mirall alternatiu per a Ubuntu, deixa'l buit i prem [done] directament.
8. 8. A la distribució d'emmagatzematge, aturem-nos-hi per assegurar-nos que entenem el que estem fent aquí (la configuració del sistema de fitxers és MOLT important perquè el teu servidor funcioni correctament). Segons el teu cas, escull una de les següents opcions:
   1. Marca "Utilitza tot el disc", si només tens una unitat d'emmagatzematge principal (1+ TB). En aquest cas, assegura't que desmarques "Configura aquest disc com a grup LVM" abans de seleccionar [done] i prémer ENTER. A continuació, continua amb el pas 9.
   2. Marca "Distribució d'emmagatzematge personalitzada", si vols utilitzar un disc secundari, per exemple, un disc principal per al sistema i un disc secundari per a dades. Assegura't que aquest disc està muntat i formateja'l si tens dubtes que encara pugui tenir dades que no necessites.

En el meu cas, vaig seguir la segona opció ja que tinc un disc intern per al sistema i algunes aplicacions, i un altre disc dur extern per a altres dades que vull mantenir separades del sistema principal en cas que li passi alguna cosa.

En el futur, hauries de veure el teu sistema de fitxers d'una manera similar a aquesta:

Observa com tenim el directori personal (/) muntat a la partició sda2, i el directori d'emmagatzematge extern (/mnt) muntat a la partició sdb1.

9. 9. Confirma l'acció destructiva seleccionant l'opció [Continua]. Prem ENTER
10. 10. Segueix seleccionant "Ometre per ara", quan aparegui la secció "Actualitza a Ubuntu Pro", prem ENTER al botó [done]
11. 11. Tria el teu nom d'usuari i contrasenya. Assegura't que no siguin fàcils d'endevinar ni contrasenyes que s'hagin filtrat a internet (ho pots comprovar a https://haveibeenpwned.com). Encara que no obrim el servidor a la xarxa pública d'internet, la seguretat mai no s'ha de menysprear.
12. 12. Prem ENTER per marcar "Instal·lar servidor OpenSSH" prement la tecla ENTER, i cap avall per seleccionar la casella [done] i prem ENTER novament
13. 13. Si vols preinstal·lar programari addicional (no recomanat, es poden instal·lar més tard), selecciona'ls, si no, prem [done] directament per saltar al següent pas.
14. 14. Ara es duran a terme totes les configuracions anteriors i s'instal·larà el sistema. Això trigarà uns minuts en funció del maquinari utilitzat. Pots mostrar registres extensos prement [View full log] si vols.
    ⌛ Espera fins que finalitzi la instal·lació, quan succeeixi, apareixerà [Reboot now]. Selecciona-ho i prem ENTER.
15. 15. Quan el missatge et mostri "Elimina el medi d'instal·lació, després prem ENTER", treu la memòria USB del PC i prem ENTER finalment.

Ja tenim el sistema operatiu instal·lat! Ara l'ordinador hauria de reiniciar-se i mostrar-te el missatge per iniciar la sessió. Pots desconnectar el teclat i la pantalla, i continuar connectant-te remotament des del teu ordinador habitual per continuar amb la instal·lació. Si no saps com fer-ho, ho explico a la següent secció 👇.

2. Accés remot per SSH

Connectar la pantalla i el teclat al servidor directament és un bon últim recurs per utilitzar-lo, però idealment volem poder accedir-hi des de qualsevol lloc. Sobretot quan som a casa, però també en cas que instal·lem algunes aplicacions que vulguem utilitzar en el nostre dia a dia, necessitem una manera de connectar-nos remotament a ell, i si és possible, de manera segura i privada.

Per això, en lloc d'utilitzar la pantalla i el teclat, ara iniciarem la sessió al nostre servidor a través d'SSH. SSH és un protocol anomenat Secure SHell, que ens permet crear una connexió xifrada i segura entre dos dispositius.

Per fer-ho, obrirem el terminal al nostre ordinador personal i escriurem la següent comanda:

ssh <usuari>@<IP_del_servidor>

L'adreça IP del servidor es mostrarà a la pantalla del servidor si encara el tens connectat. En cas contrari, el pots trobar mitjançant una aplicació de smartphone com Fing, o utilitzant un software de CLI com Netdiscover, que detecta els dispositius que estan connectats a la mateixa xarxa que tu.

IMPORTANT: Assegureu-vos que esteu connectat a la mateixa xarxa wifi que el vostre servidor personal, en cas contrari Fing serà inútil.

Un cop identificat, podeu accedir al vostre servidor a través de ssh mitjançant aquesta IP i el nom d'usuari que heu seleccionat prèviament.

Normalment les IP tenen aquest format: 192.168.XXX.XXX
Fixeu-vos en la següent imatge com al meu servidor se li ha assignat la IP 192.168.1.79 per a la interfície de xarxa en01.

Genial! Ara hauríeu d'iniciar la sessió dins del vostre servidor! Enhorabona! 😉

No obstant això, encara hi ha un petit problema. Aquesta manera de connectar-nos al servidor és genial, però només funciona si estem a la mateixa xarxa que el servidor. Per tant, ara per ara, només quan som a casa podem connectar-nos-hi.

Per això, el primer que farem al nostre servidor és blindar-lo des de fora i instal·lar una VPN privada per poder connectar-nos-hi des de qualsevol lloc sense preocupar-nos d'estar exposats a internet.

Seguim!

3. Configuracions inicials

Per assegurar-nos que les comunicacions amb el servidor siguin segures, ara aplicarem algunes mesures de seguretat i usabilitat adequades.

1. 1. Fes que el teu usuari sigui un superusuari

sudo usermod -a -G sudo <usuari>

En aquest comandament, <usuari> substitueix-ho pel nom d'usuari que has escollit durant la instal·lació.

2. 2. (opcional) Inicia la sessió amb claus SSH –>En lloc d'introduir la contrasenya del servidor cada vegada, també el podem configurar perquè només permeti que determinats dispositius s'hi connectin.

Per fer-ho, necessitem afegir la nostra pròpia clau SSH al fitxer .ssh/authorized_keys teclejant

sudo nano .ssh/authorized_keys

i afegint-hi les nostres claus SSH.

Si no tens una clau SSH, pots crear-ne una al teu ordinador personal (no al servidor) executant aquesta comanda:

ssh-keygen -t rsa -b 2048

Després, simplement copia i enganxa el contingut de la clau generada dins del fitxer authorized_keys, i estaràs a punt per iniciar la sessió autenticant-te amb la teva clau SSH privada, en lloc d'utilitzar la contrasenya.

3. 3. Com que en el nostre cas hem creat un directori muntat al disc connectat, li donarem permisos suficients al nostre usuari per accedir-hi sense problemes. En el nostre cas, el directori muntat a l'emmagatzematge secundari és /mnt.

sudo chown <usuari>:<usuari> /mnt

En aquesta comanda, <usuari> substitueix-ho pel nom d'usuari que has escollit durant la instal·lació.

4. 4. Assignarem una IP estàtica al servidor. No volem descobrir cada vegada que el connectem a una xarxa si l'adreça IP ha canviat, per això li assignarem una d'estàtica. Pots escollir la que vulguis (que encara no s'utilitzi) del rang de 192.168.1.2 a 192.168.1.254.

Per modificar la configuració de xarxa actual, necessitem editar el següent fitxer:

sudo nano /etc/netplan/00-installer-config.yaml

A continuació, afegeix les següents línies:

network: 
  ethernets: 
    eno1: 
      dhcp4: false 
      dhcp6: false 
      addresses: 
      - 192.168.1.79/24 
      nameservers: 
        addresses: 
        - 9.9.9.9 
        - 149.112.112.112 
        search: [] 
      routes: 
      - to: default 
        via: 192.168.1.1 
  version: 2

Si no vols que se't tanqui la sessió SSH actual, assegura't d'utilitzar la mateixa IP que el teu router t'havia assignat prèviament a l'IP estàtica.

Nota: També he modificat els servidors de noms per utilitzar els de Quad9 a causa de la major privadesa que ofereix. Pots escollir els que prefereixis, o simplement no afegir-los per a utilitzar els predeterminats.

Desa i tanca l'editor amb CTRL+O i CTRL+X i executa la següent comanda:

sudo netplan apply

Ara, la teva adreça IP i els servidors DNS s'haurien d'haver modificat.

Per comprovar-ho, pots executar ip a, ifconfig i resolvectl status, i hauries de veure els nous valors.

5. 5. Finalment configurarem i iniciarem el tallafoc preinstal·lat del nostre servidor Linux; UFW (Uncomplicated Firewall) amb les següents comandes:

$ sudo ufw default deny incoming 
$ sudo ufw default allow outgoing 
$ sudo ufw allow 22/tcp comment 'allow SSH from anywhere' 
$ sudo ufw logging off 
# Enable ufw, when the prompt shows you "Command may disrupt existing ssh connections. Proceed with operation (y|n)?", press "y" and enter 
$ sudo ufw enable

4. Connexió des de l'exterior (Tailscale)

Genial, gairebé hem acabat per avui, però fa falta una última cosa abans de plegar. Ara que tenim el servidor correctament protegit i configurat, seria ideal poder-hi accedir des de fora sense sacrificar la nostra privadesa. Per això, utilitzarem un programari VPN privat anomenat Tailscale. Aquesta VPN és, en essència, un túnel WireGuard que no requereix cap coneixement per configurar-lo.

En cas de tenir coneixements més elevats, i vulgueu configurar la vostre pròpia connexió amb WireGuard, endavant! Us donarà més flexibilitat en la configuració que Tailscale, tot i que aquest ofereixi certs serveis que WireGuard no fa.

El més interessant de Tailscale és que encara que sigui un servei de tercers per comunicar-nos amb els nostres dispositius personals, l'empresa no pot veure res perquè tot el tràfic està xifrat de punta a punta (E2EE; End to End Encrypted). Tailscale només ajuda els nostres dispositius a identificar-se i comunicar-se entre ells de manera segura, sense comprometre les dades que envien.

Quan vulgueu crear la vostra pròpia xarxa privada amb Tailscale, necessitareu un compte per connectar els vostres dispositius entre ells. I aquest serà el compte que utilitzareu en tots ells. Tailscale ofereix diverses opcions per iniciar la sessió. En el meu cas, utilitzo el meu compte de Github.

Per tant, anirem al seu lloc web aquí i farem clic a "Prova gratuïta". A continuació, se'ns mostrarà l'opció d'iniciar la sessió i haurem de triar quin proveïdor de serveis utilitzarem. Introduïu les vostres dades d'autentificació i haureu creat el vostre compte de Tailscale.

Instal·lació de Tailscale als nostres dispositius

Un cop dins, volem començar a afegir dispositius nous. La manera més fàcil és fent clic a "Afegeix dispositiu nou" a la part superior esquerra del tauler d'administració. Allà veureu les diferents opcions en funció del sistema operatiu del vostre dispositiu.

Per a dispositius mòbils i d'escriptori, podem descarregar les aplicacions Tailscale i seguir exactament el mateix procediment que hem fet fins ara. Els vostres dispositius començaran a aparèixer al tauler de control un cop inicieu la sessió al vostre compte de Tailscale a través d'un d'ells. Però tingueu en compte que heu d'aprovar individualment cadascun dels dispositius des del tauler d'administració.

Tanmateix, per als nostres servidors, és una mica més complicat. Hauràs d'iniciar la sessió al servidor a través de SSH la primera vegada que el vulguis connectar a Tailscale. Un cop dins, hauràs d'executar l'ordre proporcionada per Tailscale:

sudo curl -fsSL https://tailscale.com/install.sh | sh

Això instal·larà totes les biblioteques i paquets necessaris perquè Tailscale funcioni correctament. Un cop finalitzada la instal·lació, executa la següent comanda al terminal del servidor:

sudo tailscale up

Copia l'URL que et dóna a la sortida del terminal i enganxa-la al teu navegador. Inicia la sessió amb el mateix compte que vas utilitzar anteriorment, i ara tindràs el servidor connectat a través d'un túnel WireGuard amb els teus altres dispositius!

Magia! Sense obrir ports ni lluitar amb la configuració de la xarxa!

Tanmateix, afegirem al nostre tallafoc una nova regla:

sudo ufw allow in on tailscale0 
sudo ufw reload

Ara veureu que Tailscale ha assignat algunes IP als vostres diferents dispositius connectats. Aquestes IP seran les que utilitzarem per accedir a (gairebé) totes les nostres aplicacions allotjades per nosaltres mateixos. Connectar-s'hi a través de Tailscale ofereix beneficis de seguretat i privadesa, ja que tot el tràfic que passa per la vostra xarxa de malla (mesh network) Tailscale serà xifrat de punta a punta (E2EE), de manera que ningú podrà espiar el vostre tràfic encara que estigui dins de la vostre xarxa wifi.

Si ara voleu augmentar encara més la seguretat del vostre servidor permetent només l'inici de sessió mitjançant la xarxa de Tailscale, podeu seguir aquesta guia.

Consell ràpid: Després d'aprovar el vostre dispositiu nou des del lloc web del tauler de control de Tailscale, feu clic a Machine Settings > Disable key expiry. Això garantirà que el servidor SEMPRE estigui connectat a la vostra xarxa privada i no necessiteu tornar a casa per tornar-vos a autenticar cada dos dies.

5. Últim pas: reiniciar el servidor

Doncs ja ho tens! El teu nou "cloud" personal està llest per començar a instal·lar-hi aplicacions!

Ara el tenim protegit del tràfic d'internet extern i ens hi podem connectar a través de Tailscale des de qualsevol lloc del món.

Fem un reinici final i assegurem-nos que un cop tornem a iniciar la sessió tot està correcte. Si no és així, és que t'has saltat algun dels passos i hauràs de revisar quin és el problema que s'ha de resoldre abans de continuar.

Aplica el reinici amb sudo reboot now i després d'uns minuts, intenta iniciar la sessió però aquesta vegada utilitzant l'IP de Tailscale, en lloc de la de la teva xarxa local.

Si pots iniciar la sessió, Tailscale funciona bé. Ara comprova que el sistema d'arxius encara està muntat correctament amb lsblk . Hauries de veure el mateix que hem vist al principi d'aquest article:

En cas que el sistema ho necessiti, també aplica les actualitzacions necessàries. Pots comprovar-ho amb sudo apt list --upgradable

I si tens actualitzacions pendents, executa sudo apt update && sudo apt upgrade

Després d'això, només puc dir-te:
Enhorabona. Estàs en el camí correcte per convertir-te en un veritable individu soberà digital.

Notes finals

En cas de tenir qualsevol dubte, trobaràs les meves dades de contacte a la pàgina de contacte.

Si aquest blog t'ha aportat valor o has après alguna cosa nova, pots ajudar-me compartint-lo a xarxes socials, o deixant-me una donació a través de la icona del llamp que trobareu a la part inferior dreta.
Sí, aquí només s'accepta Bitcoin ⚡

In a digital era that is evolving to a surveillance nightmare, self hosting needs to become the norm… Get back to the roots, to understand the rotten leaves.

In this guide I’ll explain how to create your own home server, from scratch. Without using any pre-installed operating systems, but flashing a fresh ubuntu server into the hardware of your choice, and configuring it to have a secure and ready device to act as a home server.

As I keep adding new software into it, I’ll keep creating and uploading more guides to explain how I’ve installed and I’m using the other pieces of software. Keep tuned if you want to learn how to become a sovereign Internet Ninja, and stop giving your private data to big companies, who will just benefit from you.

Problem

We depend heavily on third-party services, both for our data and for our money. We are constantly under surveillance for our actions, and we cannot be free until we understand how to use digital services privately.

The constant surrender of our data allows large corporations to do business with them, and they have to spend more and more materials, resources, and energy on data centers to manage them.

Solution

Install services and applications on a computer hosted at home.

Stop giving your data away for other’s benefits and manage to be truly free in the digital world, without losing any advantages when using the Internet with our own electronic devices.

Give a purpose to devices that would otherwise end up in the trash, contributing to reducing waste and promoting the reuse of obsolete computer materials.

Self-hosting services and using open-source programs benefit both us and our planet.

Revolution begins with small gestures. If we understand the power we have and use it, it will never be taken away from us again.

Become a sovereign individual.

Installation

0. Preparations

Before starting to get our hands dirty, it is important that we have all the necessary material to build our own home server. Otherwise you’ll need to rush to the hardware shop to buy the weirdest adapter you can think of cause your server does not have a certain port.

Make sure you know which ports you can use with your new hardware, to connect the following pieces to it:

• Screen
• Keyboard
• Ethernet Cable
• External USB attached storage (HDD or SDD)
• A pen drive

Once we’ve got everything, we can start preparing our new setup.

Without plugging it to the wall, connect your server to your router using an Ethernet cable, attach the external storage through USB or SATA, and connect both the keyboard on a USB port, and the screen with an hdmi or display port cable.

Now, before starting your server, we will prepare the new operating system that will power your machine. In this case, we will be using Ubuntu Server 22.04. You can find the download link here: https://ubuntu.com/download/server

Download the latest version and store it locally on your computer.

Now also download (if you still don’t have it) Balena Etcher: https://etcher.balena.io/ → this software will be useful each time you want to flash an operating system to one of your devices.

Finally, plug in your pen drive, open Balena Etcher, and choose the file you just downloaded (ubuntu server .iso file) to be flashed on your pen drive.

After the flashing is completed, you can remove the pen drive, and we will be ready to start the real installation of your home server.

1. Installing the Operating System

Installation and walk-through of the steps, and special attention into using the full disk but also external disk and creating a data structure for it to be mounted to (/mnt).

BEFORE turning on your server for the first time, plug the USB pen drive that you previously flashed Ubuntu Server into it.

Now, we can start the server for the first time. Plug it to the electricity, and press the button to start it up if there is one.

Usually you will be automatically shown with a window to do different actions on the first boot. At this stage you either need press one of the keys that should be indicated to enter the boot menu, or simply go through the list of available options with your keyboard arrows, and click enter on the boot menu option.

There you’ll see the option to boot from the removable device (pen drive). Click on that one. You will now start the process of flashing the full new operating system into your home server.

Use the UP, Down, and ENTER keys of your keyboard to navigate to the options. Follow the next instructions:

1. On the first screen, select the language of your choice (English recommended)

2. If there is an installer update available, select “Update to the new installer”, press enter, and wait

3. Select your keyboard layout and variant (Spanish recommended to Spanish native speakers) and press [done]

4. Keep selecting “Ubuntu Server” as the base for the installation, down to [done], and press enter

5. Select the interface network connection that you choose to use (Ethernet recommended) and take note of your IP obtained automatically through DHCP. (Normally 192.168.x.xx). Press [done]
note that we will be assigning a static IP to the server later

6. Leave the empty next option if you don’t want to use an HTTP proxy to access it. Press [done]

7. If you don’t want to use an alternative mirror for Ubuntu, leave it empty and press [done] directly

8. On the storage layout, let’s stop a little bit more and make sure we understand what we’re doing here (the filesystem configuration is VERY important for your server to operate correctly).

8.1. Check “Use an entire disk”, if you have only one primary unit storage (1+ TB). In this case, ensure that you uncheck “Set up this disk as an LVM group” before select [done] and press enter. Then, continue with step 9.

8.2. Check “Custom storage layout”, if you want to use one secondary disk, e.g. primary for the system and secondary disk for data. Make sure this disk is mounted, and format it if you have doubts that it could still have data that you don’t need.

In my case, I followed the 2nd option as I have an internal disk for the system, and some of the apps, and another external HDD disk for other data that I want to keep separated from the main system in case something happens to it.

In the future, you should see your filesystem in a similar way to this one:

Check out how we have both the home directory (/) mounted on the sda2 partition, and the external storage directory (/mnt) mounted on the sdb1 partition.

9. Confirm destructive action by selecting the [Continue] option. Press enter

10. Keep selecting “Skip for now”, when the “Upgrade to Ubuntu Pro” section appears you press enter on the done button

11. Choose your username and password. Make sure they are not easy to guess or passwords that may have been leaked on the internet. Even if we will not be opening the server to the public internet, security should never be disregarded.

12. Press enter to check “Install OpenSSH server” by pressing the enter key, and down to select the [Done] box and press enter again

13. If you want to preinstall some additional software (not recommended, they can be installed later), select them, if not, press [done] directly to jump to the next step.

14. Now all before configurations will be applied and the system installed. This would be a few minutes depending on the hardware used. You can show extended logs by pressing [View full log] if you want.

⌛ Wait until the installation finishes, when it happens, [Reboot now] will appear. Select it and press enter.

15. When the prompt shows you “Please remove the installation medium, then press ENTER”, extract the pen drive of the PC and press enter finally.

Now the PC should reboot and show you the prompt to log in. You can disconnect the keyboard and the screen, and proceed to connect remotely from your regular computer to continue with the installation.

2. Acessing remotely (SSH)

Connecting the screen and keyboard to the server directly are a good last resource to use it, but ideally we want to be able to access it from anywhere. Specially when we are next to it at home, but also in case we install some apps that we want to use in our day to day, we need a way to connect remotely to it, and if possible, secure and privately.

That’s why instead of using the screen and keyboard, we will now log in to our server through SSH. SSH is a protocol called Secure SHell, which allows us to create an encrypted and secure connection between two devices.

To do so, we will open the terminal in our personal computer, and type the following command:

ssh <user>@<server-IP>

The server IP will be shown in your server screen if you still have it plugged. Otherwise you can find it by using an app like https://www.fing.com/products/fing-app , which detects the devices that are connected in the same network as you are.

IMPORTANT: Make sure you are connected to the same wifi as your personal server, otherwise Fing will be useless.

Once identified, you can access your server through ssh using that IP and the username you chose previously.

Usually the IP’s have this form: 192.168.XXX.XXX

Cool! Now you should be logged inside your server! Congrats 😉

However, there is still one little problem. This way of connecting to the server is cool, but only works if we find ourselves in the same network as the server. So effectively, only when we are home we can connect to it.

That’s why the first thing that we will do to our server, is blind it from the outside, and install a private VPN to be able to connect to it from the outside without worrying on being exposed to the internet.

Let’s keep going!

3. Initial Configurations

To make sure our communications with the server are secure, we will now apply some security and proper usability measures.

1. Make your user a super user

sudo usermod -a -G sudo <user>

2. (optional) Login with SSH keys

Instead of entering our server password each time, we can also configure it to ONLY allow certain devices to connect to it.

To do so, we need to add our own SSH key into the .ssh/athorized_keys by typing

sudo nano .ssh/authorized_keys

and adding our SSH keys into it. If you don’t have an SSH key, you can create one by executing this command:

ssh-keygen -t rsa -b 2048

Then simply copy paste the content of the generated key inside the authorized_keys file, and you’ll be ready to log in authenticating with your private SSH key, instead of using the password.

3. Since in our case we have created a directory mounted in the attached disk, we will give enough permissions to our user to access it without problems. In our case, the mounted directory in the secondary storage is /mnt.

sudo chown <user>:<user> /mnt

4. We’ll start with assigning a static IP to our server. We don’t want to discover each time we connect it to a network if the IP has changed, so that’s why we will assign a static one to it. You can choose the one you want (that is not already in use) on the range from 192.168.1.2 to 192.168.1.254.

To modify the current network configuration, we need to edit the following file:

sudo nano /etc/netplan/00-installer-config.yaml

And add the following lines:

network: 
  ethernets: 
    eno1: 
      dhcp4: false 
      dhcp6: false 
      addresses: 
      - 192.168.1.79/24 
      nameservers: 
        addresses: 
        - 9.9.9.9 
        - 149.112.112.112 
        search: [] 
      routes: 
      - to: default 
        via: 192.168.1.1 
  version: 2

If you don’t want to break your current ssh session, make sure to use the same IP that you were assigned previously by your router to the static IP.

Note: I’ve also modified the nameservers to use the Quad9 ones due to higher privacy in their end. You can choose the ones you prefer.

Save and close the editor, and run the following command:

sudo netplan apply

Now your IP address and DNS servers should be modified.

To check it you can either run ip a , ifconfig and resolvectl status , and you should see the new values there.

5. Finally we will configure and start the pre-installed firewall from our linux server: UFW (Uncomplicated Firewall).

Run the following commands:

$ sudo ufw default deny incoming 
$ sudo ufw default allow outgoing 
$ sudo ufw allow 22/tcp comment 'allow SSH from anywhere' 
$ sudo ufw logging off 
# Enable ufw, when the prompt shows you "Command may disrupt existing ssh connections. Proceed with operation (y|n)?", press "y" and enter 
$ sudo ufw enable

5. Connecting from the outside (Tailscale)

Alright, we’ll be almost done for today, but there is one last thing missing.

Now that we have our server properly secured and configured, it would be ideal to be able to access it from the outside without sacrificing our privacy.

That’s why we will be using a private VPN software called Tailscale. This VPN is effectively a WireGuard tunnel that requires 0 knowledge to set up.

The cool thing, is that even if we use a 3d party service to communicate with our personal devices, they cannot see anything cause all the traffic is End to End Encrypted (E2EE). Tailscale just helps our devices identify them, and communicate between them securely, without compromising the data that they are sending.

When you want to create your own private network with Tailscale, you’ll need to use an account to connect to your devices between them. And this will be the account that you’ll use in all of them.

Tailscale offers a variety of different options to log in. In my case I’m using Github.

So we’ll go to their website https://tailscale.com and click on “Try for free”.

Then we’ll be prompted with the option to log in, and we’ll need to choose which service provider we’ll be using.

Enter your authentication details, and you’ll have created your tailscale account.

Installing Tailscale on our devices

Once inside, we want to start adding new devices.

The easiest way is by clicking on “Add new device” on the top left part of your admin dashboard. You’ll see there the different options depending on your device operating system.

For mobile, and desktop clients, we can download the Tailscale apps, and simply go through the same procedure we’ve done up until now.

Your devices will start appearing in your dashboard as soon as you log in to your tailscale account through one of them. But take into account that you must individually approve each one of the devices from your administration dashboard.

However, for our servers, it’s a little bit more tricky. We will need to log in to our server through ssh the first time we want to connect it to tailscale.

Once we’re inside, we will need to execute the command provided by tailscale:

sudo curl -fsSL https://tailscale.com/install.sh | sh

This will install all the libraries and packages needed for tailscale to run properly. After the installation is complete, run the following command in your server’s terminal:

sudo tailscale up

Copy the URL that it gives you in the terminal output, and paste it in your browser. Log in with the same account you used previously, and you’ll now have your server connected through a WireGuard Tunnel with your other devices!

Voilà, without opening ports, or fighting with your network settings!

However, we will indicate to our firewall a new rule:

sudo ufw allow in on tailscale0 
sudo ufw reload

Now you’ll see that Tailscale has assigned some IPs to your different connected devices. This IPs will be the ones we will use to access (almost) all our self-hosted applications. Connecting to them through Tailscale brings security and privacy benefits, as all the traffic that goes through your mesh Tailscale network, will be E2EE, so no one can eavesdrop on your traffic even if they are inside your home network.

If you would like now to even increase the security of your node by only allowing to log in using Tailscale, you can follow this guide: https://tailscale.com/kb/1077/secure-server-ubuntu-18-04/

Quick tip: After approving your new device from the Tailscale dashboard website, click on Machine Settings > Disable key expiry. This will ensure that the server is ALWAYS connected to your private network, and you don’t need to go home to reauthenticate every a couple of days.

6. Final Check — Rebooting

There you are, your new personal home server is ready to start installing apps into it!

We now have it protected from the outside internet traffic, and we can connect to it through Tailscale from anywhere in the world.

Let’s make a final reboot, and make sure that once we log in again everything is correct. If not, you missed some of the steps and should check out what’s the problem that needs to be solved before proceeding!

Apply the reboot with sudo reboot now and after some minutes, try logging in but this time using the Tailscale IP, instead of your local network one.

If you manage to log in, Tailscale is working fine. Now check that the filesystem is still correctly mounted with lsblk and in case the system needs it, apply the necessary updates.

After this, I can only say: Congrats. You’re on the correct path of becoming an Internet Ninja.

In graph theory, the shortest path problem is the problem of finding a path between two vertices (or nodes) in a graph such that the sum of the weights of its constituent edges is minimized. Common algorithms for solving the shortest path problem include the Bellman-Ford algorithm and Dijkstra’s algorithm.

Bellman-Ford

This algorithm aims to find the shortest path to go from a source (s) to any destination in a certain topology.

Topologies are composed of vertex (nodes) and edges (links).

There are 2 different versions of topologies. Centralized ones, where the source node knows the whole topology, and distributed ones. In the latest version, a node wants to compute the shortest path to other nodes, but without knowing the whole topology, the source node will only have the estimated distances provided by its neighbors.

To explain it in plain words, the neighbor nodes exchange their vector of distances between them:

When a node u receives the vector of distances dk from its neighbor k. it can use the distances on this vector to relax the edge to this neighbor (u,k).

Let’s see an example of a topology that uses the distributed version of BF to show the distribution of the route to the node a. You’ll see the update messages and the route to the node a that each node learns when the distance vectors are exchanged between them.

Notice the decentralized computation: d sends only the best path to a.

Process:

• b relaxes (a,b) and sends the update to its neighbors.
• d receives the previous update from b and then relaxes (d,b) and sends the update to its neighbors.
• g receives the previous update from d and then relaxes (g, d) and sends the update to its neighbors.
• h receives the previous update from g and relaxes (h,g).

See how the algorithm converges to the optimal solution (shortest path) because this decentralized process? It runs BF in an end-less loop in which each node relaxes its edges.

Once all nodes have sent and received update messages it turns out to be like an iteration of the centralized algorithm.

Dijkstra

This is another algorithm to find the shortest path to go from a source (s) to any destination in a certain topology.

The difference between this one and BF, is that Dijkstra only works in topologies that are free of edges with negative weights.

Let’s see an example of how Dijkstra algorithm works in the following topology:

When we initialize the algorithm, there are no sources, and the only “reachable” destination is node1.

• In the first iteration, we find the distance between 1–2 and 1–3. The path cell tells us that the only “shortest path” available now is how to reach node1 (only 1 hop without weight).
• Extract 2 shows that we do now have 2 sources to reach the nodes 3, 4 and 5, and that the shortest path uses nodes 1 and 2 to reach node 5.
• Extract 5 has node 5 as a source now. However instead of going for node4, it detects that now the shortest path (the one with less weight) is node3, so it adds the distance to reach node3 from node1, instead of adding the distance to the direct neighbor from the previous extract.
• Extract 3 is the last one that is looking for a route to reach the only missing node (4). It calculates that the shortest path to reach node 4 is through nodes 2 and 5, instead of node 3.
• Finally we do have all the distances between the nodes from the topology, and the shortest paths to reach each one of them.

Great, now that you have had a quick glimpse on SPAs, let’s get into the real matter of this article. Routing Information Protocol (RIP).

2. RIP

The Routing Information Protocol is a dynamic routing protocol that is used in small/medium IP networks. It is based on a distance-vector exchange, and a distributed version of BF.

RIP has been evolving during the years, and new versions have been surging, like RIPv2, RIPng (RIP next generation), which has been adapted for IPv6, and lately another protocol called OSPF has been gaining popularity. However, to understand all of the above, we need to know how the basic form of routing works.

RIP, like any other routing protocol, defines a routing database, a protocol for exchanging information about routes and an algorithm for updating routing information.

We will describe these three parts below:

2.1 Routing Database

Each RIP entity (normally routers) keep track in its routing database of all networks (and possibly individual hosts) in the RIP routing domain.

Each entry in the routing database includes the next intermediary router (called next hop) to which datagrams have to be delivered so that they can reach the final destination. In addition, the routing database includes a metric for measuring the total distance to the final destination.

The fields specified in the routing database are:

• Address. Destination network or host. (@IP/MASK)
• Metric. The metric or cost from that node to the destination.*
• Router. Next-hop to reach the destination network or host.
• Interface. The network interface that must be used to reach the next router.
• Timers. Used to manage dynamics of the routing information.

*The metric is basically the number of hops to the destination. A datagram makes a hop when it goes through a router. For example, if a RIP entity is directly connected to its destination, the distance is 1 hop, but if the source and destination are connected through another intermediary router, then the distance is 2 hops.

The valid metrics range is between 1 and 16 hops, however the maximum number of hops allowed for any destination is 15, and the distance 16 is reserved for “infinity”, in other words, destination unreachable.

2.2 Updating Algorithm

This is the part where RIP gets fun (at least for coders/technical people ;) ).

Distance vectors algorithms get their name from the fact that it is possible to compute optimal routes by periodically exchanging the vector of distances to the different destinations that each node in the network has.

The Routing Database of each RIP node is initialized with a description of the RIP entities that are directly connected (next-hop=1), and then it updates with a distributed version of BF.

However, what happens if the network we are in changes, a RIP entity disconnects, disappears, or another entity gets inside the network and then there is a better optimal path?

Let’s first explain how the algorithm behaves in a static topology, and then we’ll dive deeper into dynamic topologies and the problems that we may find.

Static Topology

Let’s first define some variables. w(i,k) is the weight or cost of the edge that connects nodes i and k. So w(i, k) = 1 if i and k are directly connected and w(i, k) = 16 (infinity) if they are not. Moreover, m(P(i,j)) as the best current metric for the path or route between i and j.

The Bellman-ford equation and it says that “the best route is through the neighbor that has the minimum distance to the destination”.

So, when a RIP entity i receives the distance vector dk with the estimates of neighbor k, it adds w(i, k) to each of the estimations received. Then, for each destination n, the node i compares the metric provided by the neighbor with its current routing entry metric for this destination R(n).metric. The node picks the new route if the metric provided by the neighbor is smaller. With this algorithm, after receiving estimates from all the nodes in the network, i will have the smallest distance to all the destinations.

Dynamic Topology

Decrease the metric

The method so far only has a way to lower the metric, as the existing metric is kept until a smaller one shows up.

However, it is possible that the initial estimate might be too low. In this case, we need a method for “increasing the metric”.

For this purpose, it is enough to always consider the information received by the next hop of a route. For example, suppose the current route to a destination has metric D and uses router R. If a new set of information arrived from some source other than R, only update the route if the new metric is better than D. But if a new set of information arrives from R itself, always update D to the new value.

Updates

It is safe to run the algorithm asynchronously, that is, each RIP entity can send updates with its distance vector according to its own clock. The algorithm will converge to the correct distances in finite time in the absence of topology changes.

Originally each RIP router transmitted full updates every 30 seconds. In the early deployments, routing tables were small enough that the traffic was not significant. As networks grew in size, however, it became evident there could be a massive traffic burst every 30 seconds, even if the routers had been initialized at random times.

Modern RIP implementations introduce deliberate variation into the update timer intervals of each router.

Example

As you can see, N1 is connected to RA and RB. We are going to illustrate how the information about Network 1 (N1) can be distributed by RIP. As the order of the RIP updates is randomly distributed (unpredictable), the description that follows is just a possible realization of the RIP update process.

Let’s go step by step:

(1) As initial condition we assume that the only routers that have information about N1 are the two directly connected routers (RA and RB). Then, RA is the first router to send information about N1 in a RIP message (in this case, but RB could also be the first one). We will note this information as {N1,1}, which means that the RIP message sent by RA includes an entry for N1 showing that this router can reach this network with one hop. RA sends this RIP message to its one-hop neighbors: RB and RC. Upon receiving this information, RC updates its routing database because the information received by RA informs about a new reachable network. RB does nothing because it already knows N1 with a better (shorter) path.

(2) RC decides that it has to send a RIP message and it includes the new information it knows about N1. This information is that it can reach N1 with two hops {N1,2}. RC sends this information to its one-hop neighbors: RA and RD. In the case of RD, this is the first time it hears about N1, so it updates its routing database with the new information. Obviously, RA does nothing.

(3) This time RB sends its RIP message including the entry {N1,1} to its one-hop neighbors: RA and RD. RA does nothing, but RD updates its routing database because the new information from RB provides a shortest path to the destination than the previous one that RD possessed.

(4) This time RD sends its RIP message but nobody does nothing because the information provided by the router is worse than the information present at the routing databases of the rest of the routers (this is logical because RD is the farthest router).

BROKEN LINK

Now, we are going to assume that one of the links is broken. In particular, the link that connected RB to N1.

In this case, we are going to observe that some nodes have too low estimations for N1 and we will see how nodes use the method for increasing the metric. The description that follows is a possible realization of the RIP update process:

(1) As an initial condition, we assume that all the routers have converged to the topology depicted in the first example. Thus, they have entries for N1. Then, RB detects that its link to N1 is broken. At this moment, RB updates the entry for N1 in its database setting the metric for this network to infinity (16).

(2) RA sends a RIP message to its one-hop neighbors: RB and RC. Upon receiving this information, RB updates its routing database because the information received by RA informs about a new path to reach again the network N1. RC does nothing because it already knows N1 with the same metric.

(3) At this time RB sends a RIP message to its one-hop neighbors: RA and RD. RA does nothing, but RD updates its routing database because although the information received contains a worse metric, it comes from the next hop router. Recall that according to the “increasing metric method”, we always have to update our routing entries when the information comes from the next hop. Notice also that this can be interpreted as an indication that our estimate for N1 is currently too low. Indeed, with the new network topology, RD cannot reach N1 with just two hops.

2.3 Improvements

In practice, routers and lines often fail and come back up. To properly handle dynamisms the algorithm presented so far is not yet suitable and some enhancements are required.

1. TIMERS

Notice that if a certain router X is included in the best route to a certain destination of some other router Y , and the router X is no longer available (for example because it crashed or because some network connection to it is broken), the algorithm explained so far might never reflect the change to router Y . The algorithm as shown so far depends upon routers notifying its neighbors if their metrics change. In order to handle problems of this kind, distance vector protocols must make some provision for timing out routes. For this purpose, there are two timers associated with each route: a “timeout” and a “garbage-collection” time.

• The timeout is used to limit the amount of time a route can stay in a routing database without being updated. Recall that RIP entities have to send update messages approximately every 30 seconds. The timeout is initialized to 180 seconds whenever a new route is established and is reset to the initial value whenever an update is heard for that route. If an update for a route is not heard within that 180 seconds (six update periods), the hop count for the route is changed to 16, marking the route as unreachable.
• The other timer, the garbage collection timer, is used to make help neighbors making them know that the route is no longer valid. An unreachable route will be advertised with the infinite metric (16) until the garbage-collection timer expires (120 seconds by default). After this, the route is removed from the route database.

2. COUNT TO INFINITY

To illustrate this problem, we continue our example but this time we “break“ the link between RA and N1. As a result is broken, N1 is now unreachable for our four RIP routers.

(1) As shown in the image above, at a certain moment the last link to N1 is broken. At this moment, RA, who is the only one-hop RIP neighbor of N1 detects this situation and sets the metric for this network to infinity in its routing database.

(2) At a certain moment, RB sends a RIP message with its current information about N1. That is to say, RB sends {N1,2} since it has not received any new information about N1. This out-of-date information arrives to RA since it is a one-hop neighbor of RB, and it causes the update of the RA’s entry for N1.

(3) RA sends its RIP update message including {N1,3}. This causes an update in the entries for N1 in RB and RC.

(4) RC sends its RIP update message including {N1,4}, which does not cause any update in the neighbors.

(5) RB sends its RIP update message including {N1,4}, which causes that RA increases the metric for this network up to 5. At this moment, RA and RB are in mutual deception because each mutual RIP update message causes an increase of the metric for the N1 network in the other neighbor.

Notice that the behavior of algorithm is correct in the sense that the network is now at a distance of infinity and in fact, with the successive updates, the metric for the route is slowly increasing to infinity. However, we have a problem because the “counting to infinity” will never end and thus the routing databases will never converge. Thus, at this point it might become clear why we have to limit the maximum number of hops of a RIP domain and why “infinity” should be chosen as small as possible. Notice however, that infinity must be large enough so that no real route is that big. Therefore, the choice of infinity is a trade-off between network size and speed of convergence in case counting to infinity happens. The designers of RIP believed that the protocol was unlikely to be practical for networks with a diameter larger than 15, and thus they decided to set infinity to 16.

3. SPLIT HORIZON

The counting to infinity problem that we saw in our previous example is caused by the fact that RA and RC are engaged in a pattern of mutual deception. Each claims to be able to get to N1 via the other. This can be prevented in many cases by being a bit more careful about which information is sent to which neighbors.

In general, the idea of providing a ”split view“ of your available information receives the name of ”split horizon”. In the context networking, split horizon is used to solve several problems and also to provide certain functionalities. In the context of RIP, in its simplest version, the split horizon rule is just to omit routes learned from one neighbor in updates sent to that neighbor. The reason for this, is that it is never useful to send information about a certain route to the neighbor that you are using as next hop for the route in question. Let illustrate this technique with an example:

(1) At a certain moment the last link to N1 is broken. At this moment, RA, who is the only one-hop RIP neighbor of N1 detects this situation and sets the metric for this network to infinity in its routing database.

(2) At a certain moment, RC sends a RIP message with its current information about N1 but in this case RC applies the split horizon rule and thus, it sends information about N1 only RD and not to RA because RA is the next hop for N1 in the RB’s routing database. This avoids contaminating the routing entry for N1 of RA with out-of-date information coming from RC.

(3) RA sends its update to its neighbors RB and RC. After receiving the update, RB and RC update their routing databases because both had RA as next hop router (they apply the “increasing the metric rule”).

(4) Using the simple split horizon rule, RB sends its update to RD. The final result is that RD updates its routing entry for N1 and that all the routers in the RIP domain know now that N1 is unreachable.

The simplest version of split horizon prevents the majority of the situations of mutual deception. However, in the eventual situation in which a router, say RC, thinks that it can get a network, say N1, via another router, say RD, and RD thinks that it can get N1 via RC, in this case, we have a loop. For solving this situation, the standard of RIP proposes a modification in the behavior of split horizon called “split horizon with poisoned reverse”. With poisoned reverse, there is also an “split view” of routing information available, but the idea is not omitting routes learned from one neighbor in updates sent to that neighbor but include them with their metrics set to infinity.

3.1 Split Horizon with poisoned reverse

Let’s see a particular way in which we might arrive to a situation of mutual deception although we have activated split horizon. Then, we show how activating split horizon with poison reverse might help. For this example, we are considering that packets that convey the RIP update data may suffer delays.

These delays are typically due to the process of transmitting the data through the network and also to the time spent by the packets at different queues at the sender and the receiver. These delays also cause that RIP update messages sent to a neighbor are not immediately received and processed by that neighbor.

Let’s describe the situation:

(1) As mentioned, we consider at a first moment that only the simple split horizon is activated. Then, at a certain moment, the last link to N1 is broken. At this moment, RA, who is the only one-hop RIP neighbor of N1 detects this situation and sets the metric for this network to infinity in its routing database.

(2) RA sends RIP update messages containing the entry {N1,16} to neighbors RB and RC. As it is shown in Figure 2.5, these update messages arrive to the neighbors at different instants of time.

(3) RB after receiving and updating its entry for N1 to infinity, RB sends a message to its neighbor RD. Notice that RB does not send any update message to RA since slit horizon is activated.

(4) RD sends an update to RC including {N1,3}, just a little bit before receiving the update for N1 from RB. As split horizon is activated, RD does not send information about N1 to RB. The update from RD is received by RC after the update from RA, and thus, the entry in the RC’s database for N1 is {N1,4,RD}.

(5) RC sends and update about N1 to RD just before receiving the update from RA telling that N1 is unreachable. As a result, RC will end with an entry for N1 that is {N1,3,RC}. At this point, RC and RD are in mutual deception. Furthermore, they will not send to each other any update message because of the split horizon rule. Therefore, to clean the route N1 from the routing tables, we will have to wait to the timeout.

(6) In the final step, we see that if we activate split horizon with poison reverse, RD might send an update message to RC and remove the loop with RC before the timeout for the route expires.

Certainly it is rather improbable that with simple split horizon activated two routers end with routes pointing at each other, but it can happen.

In general, split horizon with poisoned reverse is safer than simple split horizon because advertising reverse routes with a metric of 16 will break any loop of mutual deception between two routers immediately. If the reverse routes are simply not advertised, the erroneous routes will have to be eliminated by waiting for a timeout.

4. TRIGGERED UPDATES

Split horizon with poisoned reverse will prevent routing loops that involve two routers. However, it is still possible to end up with patterns in which three routers are engaged in mutual deception.

Triggered updates are used to speed up convergence by avoiding that out-of-date updates produce patterns of three or more routers in mutual deception. To implement triggered updates, we simply add a rule that whenever a router changes the metric for a route, it is required to send update messages to its neighbors almost immediately, even if it is not yet time for a regular update message.

(1) After the link to N1 is broken, RA sets the metric for this network to infinity in its routing database.

(2) RA sends an update about N1 to its neighbors RB and RC. Since this update message causes a change of metric for the entries of network N1 in RB and RC, these routers must “immediately” send a triggered update for this network.

(3) In particular, we can observe that after RB sends its triggered update and this message is processed, all the routers in the RIP routing domain have already converged to the correct metric (16) for N1.

Notice that if the system does not send any regular update while the triggered updates are being sent, it would work perfectly.

Unfortunately, things are not so nice. While the triggered updates are being sent, regular updates may be happening at the same time. Routers that haven’t received the triggered update yet will still be sending out information based on the route that no longer exists. It is possible that after the triggered update has gone through a router, it might receive a normal update from one of these routers that hasn’t yet gotten the word. This could reestablish an out-of-date version of the faulty route. If triggered updates happen quickly enough, this is very unlikely. However, counting to infinity is still possible.

The final aspect about triggered updates to be taken into account is related to performance. Triggered updates can cause excessive load on networks with limited capacity or networks with many routers on them. This is due to the fact that triggered updates cause a lot of network traffic in a short period of time. Therefore, the protocol requires that implementors include some mechanisms to avoid these performance problems. To this respect, the standard proposes two mechanisms:

• The first mechanism is to limit the frequency of triggered updates. After a triggered update is sent, a timer should be set for a random interval between 1 and 5 seconds. If other changes that would trigger updates occur before the timer expires, a single update is triggered when the timer expires. The timer is then reset to another random value between 1 and 5 seconds. Furthermore, a triggered update should be suppressed if a regular update is due by the time the triggered update would be sent.
• The second mechanism says that triggered updates do not need to include the entire routing table. In principle, only those routes which have changed need to be included. Therefore, messages generated as part of a triggered update must include at least those routes that have their route change flag set. Split Horizon processing is done when generating triggered updates as well as normal updates. The only difference between a triggered update and other update messages is the possible omission of routes that have not changed. The remaining mechanisms must be applied to all updates.

5. HOLD DOWN TIMER

As a final enhancement, we will explain the Hold Down Timer mechanism:

Each router starts the hold-down timer when it first receives information about a network that is no longer reachable (RIP distance=16). Until the hold-down timer expires, the router will discard any subsequent update messages that indicate the route is again reachable. A typical hold-down timer ranges from 60 to 120 seconds. The main advantage of the hold-down timer is that a router will not be confused by receiving spurious information about a route being accessible, when it was just recently told that the route was no longer valid. This provides a period of time for out-of-date information to be flushed from the system. However, this has a disadvantage because the hold-down timer forces a delay in a router responding to a route once it is fixed. For example, let us suppose that a network “hiccup” causes a route to go down for five seconds. After the network is up again, the hold-down timer must expire before the router will try to use that network again. This makes using hold-down relatively slow to respond and may lead to delays in accessing networks that fail intermittently.

2.4 RIP History

Great, now we’ve seen the different mechanisms that the RIP uses. Let’s now explain how does the protocol itself, and the main improvements made in each version.

RIP messages are sent using the User Datagram Protocol (UDP) with UDP port number 520 for RIP-1 and RIP-2, and 521 for RIPng. Notice that even though RIP is considered part of layer three, in terms of message exchange, RIP behaves like an application (using UDP/IP). The format of RIP messages is version-dependent. On the other hand, RIP messages can be either sent to a specific RIP neighbor (unicast), or they can be sent to multiple neighbors (broadcast or multicast). For the three versions of RIP, we have only two basic types of messages:

• RIP Requests. Requests are messages sent by a RIP entity to another RIP entity asking it to send back all or part of its routing table.
• RIP Responses. Responses are messages sent by a RIP entity containing all or part of its routing table. Despite the name “response”, as we have already seen, these messages are sent most of the time without any preceding request.

RIP-1

RIP-1, the original specification of RIP uses classful network addresses because the message format of RIP-1 does not consider sending masks. As a result, this protocol lacks support for subnetting or supernetting. Another limitation of RIP-1 is that there is not support for router authentication, making RIP vulnerable to various attacks.

RIP messages are sent using the UDP/IP network. Regarding the IP layer, the RIP-1 entity can select a unicast transmission by setting the destination IP of the neighbor or a broadcast transmission by setting the universal broadcast IP address 255.255.255.255. Regarding the UDP layer, RIP-1 uses the UDP reserved port number 520. The UDP port numbers in RIP-1 are used as follows:

• RIP Request messages are sent to UDP destination port 520. They may have a source port of 520 or may use an ephemeral port number.
• RIP Response messages sent in reply to an RIP Request are sent with a source port of 520, and a destination port equal to whatever source port the RIP Request used.
• Unsolicited RIP Response messages (sent on a routine basis and not in response to a request) are sent with both the source and destination ports set to 520.

RIP-2

RIP-2 represents a very modest change to the basic Routing Information Protocol. The new features introduced in RIP-2 are described as “extensions” to the basic protocol. The five RIP-2 extensions are:

1. Classless Addressing Support and Subnet Mask Specification. RIP-2 adds explicit support for subnets by allowing a subnet mask within the route entry for each network address. RIP-2 provides support for fixed-length subnet masking (FLSM), variable-length subnet masking (VLSM) and classless addressing (CIDR).
2. Use of Multicasting. To help reduce network load, RIP-2 allows routers to be configured to use multicast with the address 224.0.0.9.
3. Next Hop Specification. The immediate next hop IP address to which packets to the destination specified by this route entry should be forwarded. Specifying a value of 0.0.0.0 in this field indicates that routing should be via the originator of the RIP advertisement. An address specified as a next hop must, per force, be directly reachable on the logical subnet over which the advertisement is made. The purpose of the Next Hop field is to eliminate packets being routed through extra hops in the system. It is particularly useful when RIP is not being run on all of the routers on a network.
   Note that Next Hop is an “advisory” field. That is, if the provided information is ignored, a possibly sub-optimal, but absolutely valid, route may be taken. If the received Next Hop is not directly reachable, it should be treated as 0.0.0.0 .
4. Authentication. RIP-2 provides an optional authentication scheme, which allows routers to ascertain the identity of a router before it will accept RIP messages from it.
5. Route Tag. Each RIP-2 entry includes a Route Tag field, where additional information about a route can be stored. This information is propagated along with other data about the route.

RIP-2 messages are exchanged using the same basic mechanism as RIP-1 messages, that is to say, using the UDP/IP network. However, to help to reduce the network load, RIP-2 allows routers to be configured to use multicast instead of broadcast for sending out unsolicited RIP Response messages. In this case, UDP/IP datagrams are sent out using the special reserved multicast address 224.0.0.9 . All routers on a RIP-2 domain must use multicast for this feature to work properly.

RIPng

RIPng is the IPv6-compatible version of RIP for IPv6. RIPng, which is also occasionally seen as RIPv6 for obvious reasons, was designed to be as similar as possible to the current version of RIP for IPv4, which is RIP Version 2 (RIP- 2).

Despite this effort, it was not possible to define RIPng as just a new version of the older RIP protocol because of the change in the length of the addresses: from 32-bit in IPv4 to 128-bit addresses in IPv6. This forced a new message format for RIPng. The main differences between RIPv2 and RIPng are:

• Support of IPv6 networking.
• The maximum number of RTEs in RIPng is not restricted to 25 as it is in RIP-2 (and also RIP-1). It is limited only by the maximum transmission unit (MTU) of the network over which the message is being sent.
• While RIPv2 supports authentication, RIPng does not include its own authentication mechanism. It is assumed that if authentication and/or encryption are needed, they will be provided using the standard IPSec features defined for IPv6 at the IP layer. This is more efficient than implementing authentication for each individual protocol.
• RIPv2 allows attaching arbitrary tags to routes, RIPng does not.
• RIPv2 encodes the next hop into each route entries, RIPng requires specific encoding of the next hop for a set of route entries. Due to the large size of IPv6 addresses, including a Next Hop field in the format of RIPng RTEs would almost double the size of every entry. Since Next Hop is an optional feature, this would be wasteful. Instead, when a Next Hop is needed, it is specified in a separate routing entry.

RIPng uses multicasts for transmissions, using reserved IPv6 multicast address FF02::9. Since RIPng is a new protocol, it cannot use the same UDP reserved port number 520 used for RIP-1/RIP-2. Instead, RIPng uses well-known port number 521. The semantics for the use of this port is the same as those used for port 520 in RIP-1 and RIP-2.

2.5 Limitations of RIP

• The protocol is limited to 15 hops. The designers of RIP believe that the basic protocol design is inappropriate for larger networks. Note that this statement of the limit assumes that a cost of 1 is used for each network. This is the way RIP is normally configured. If the system administrator chooses to use larger costs, the upper bound of 15 can easily become a problem.
• RIP depends on “counting to infinity” to resolve certain unusual situations. If the system of networks has several hundred networks, and a routing loop was formed involving all of them, the resolution of the loop would require either much time (if the frequency of routing updates were limited) or bandwidth (if updates were sent whenever changes were detected). Such a loop would consume a large amount of network bandwidth before the loop was corrected. However, various precautions are taken that should prevent these problems in most cases.
• RIP uses fixed “metrics” to compare alternative routes. It is not appropriate for situations where routes need to be chosen based on real-time parameters such a measured delay, reliability, or load.

This article was intended to cover a basic explanation of how does RIP work on Linux systems, using the Quagga implementation.

However, as it is getting too long, I guess this is enough for an overall understanding on how RIP works, and in case you want to dig further into how it actually works in practice in Linux systems, do not doubt on contacting me.

If I get many petitions about the subject I may end up writing an article about it ;)

I’ve previously mentioned it, but the implementations of the routing protocol explained above are not at all the only ones available, and actually nowadays they aren’t either the most used, however from my point of view, they are the most begginer-friendly implementations, and once you’ve understood these, you can pass onto newer protocols like OSPF, BGP, etc.

Well well well dear readers, this is the final article of the INTERNET EDUCATIONAL SERIES which I’ve been writing about for the past winter. I hope you enjoyed it and learnt something, and in case you want to discuss in the topics explained here, or just want to chat a little bit, do not doubt on contacting me at akakush19@protonmail.com, I’ll be more than happy to answer.

Today we will talk about the Multicast technology, how do we send the same datagrams to a certain group of devices without having to send them to everyone (broadcast).

To understand Multicasting we first need to understand Unicasting.

In unicasting, there is a single sender (source) and a single receiver (destination).

When unicast routing, the router forwards the received packet through only one of its interfaces.

On the other hand, multicasting is when there is a single sender and multiple receivers. The several receivers form part of a multicast group.

When multicast routing, the router forwards the received packets through several of its interfaces.

However, we did already know what broadcasting was (sending to everyone reachable), so how do we differentiate between them?

Receivers need to request forwarding to be part of the multicast group.

This way routers know where they need to send their multicast packets.

When we want to use Multicast and send a packet from a source, the packet must include the normal source address, and a specific multicast address specified as the destination.

Moreover, the outgoing network interface and the TTL of that packet need to be specified too.

If we are one of the receivers of the multicast group, we must indicate that we want to get inside that group until we do not want to be part of it no more.

We can do so with some commands that we will see later on, but the actions that need to be performed for all multicast receivers are the following:

• Join-IP-Multicast-Group (group-address, interface)
• Leave-IP-Multicast-Group (group-address, interface)

Before digging deeper into the protocol, this are some examples of Applications of Multicasting:

One to many

• Scheduled audio-video distribution (TV, meetings, lectures…)
• File distribution and caching (website contents, file-based updates…)
• Monitoring (stock prices, telemetry)

Many to many

• Multimedia conferencing (whiteboards, audio/video)
• Concurrent processing (distributed parallel processing)
• Collaboration (shared document editing)
• Distance learning (one-to-many + feedback)
• Chat groups
• Multiplayer games

Multicast Addressing

Link-Layer Transmission/Reception

How are Transmissions?

• An IP multicast packet is transmitted as a link-layer multicast, on those links that support multicast.
• The link-layer destination address is determined by an algorithm specific to the type of link.

How are Receptions?

• Necessary steps are taken to receive desired multicasts on a particular link, such as modifying address reception filters on LAN interfaces
• Multicast routers must be able to receive all IP multicasts on a link, without knowing in advance which groups will be used.

So, we need to modify certain fields to be able to use multicast. Let’s see which ones and how do we modify them.

Address Structure

As you know, IP addresses have 32 bits. These 32 bits are splitted into two parts:

• NetID
• HostID

If the destination @IP has the same NetID as mine, we know that this host is in my same data-link network, and the HostID identifies the host within each network.

Additionally, you know that IPv4 addresses have 5 possible divisions:

• Class A: 0.0.0.0 → 127.255.255.255 (Network 10 is private and Network 127 is internal)
• Class B → 128.0.0.0–191.255.255.255 (Private addresses 172.16.0.0–172.31.255.255)
• Class C → 192.0.0.0–223.255.255.255 (Private addresses 192.168.0.0–192.168.255.255)
• Class D → 224.0.0.0–239.255.255.255 MULTICAST ADDRESSES
• Class E → 240.0.0.0–255.255.255.254 (Reserved for future use)

So yeah, until now we did talk only about classes from A to C, but of course the remaining ones are important, and in today’s article Class D takes the podium place.

Multicast addresses are inside the range 224.0.0.0 to 239.255.255.255.

However it is impossible to add all the hosts inside this range, so to solve this, hosts join multicast groups, and then they are identified inside that groups.

How are hosts identified inside a multicast group? By SIMPLE MAPPING.

The devices are identified with their MAC address, and to indicate that these hosts have joined a multicast group, we do MAP directly the multicast identifier onto the 23 least significant bits of the MAC address. Take a look at the following picture:

When a router fins a destination address that forms part of a multicast group, it will know that it has to send the packet to that specific group.

To identify how/where to send a certain packet when it is received, the MAC layer (Ethernet interface) works as follows:

As you can see, there are several steps that need to be executed before checking if the packet is part of a multicast group.

IGMP

How do hosts tell the multicast group that they want to join it?

This is when IGMP (Internet Group Management Protocol) comes into play.

When hosts want to either join, leave or indicate that they want to continue being part of a certain group, they must use IGMP messages.

Routers also use this messages to discover group members.

The different types of IGMP messages are the following:

Membership Report

Used when a host or router can join a group.

Each host maintains a list of processes that have interest in a group.

Join operation:

• When a process wants to join a new group, it sends its request to the host.
• The host then adds the name of the process and the name of the requested group to its list.
• The host sends the membership report to the router (for fiability purposes it can be sent twice).

Leave Report

When a host sees that no process is interested in a specific group, it sends a leave report.

If a router receives a leave report it will not purge the list if there are still other hosts interested in that group.

For that purpose, routers send a special query message with a specified response time for the group in question to see if there is anyone interested in that group.

If there is no response, it purges the list.

General Query Message

Membership report and leave report are not enough to maintain the membership information.

To keep a host inside a group, hosts and routers periodically send general query messages.

Then hosts and routers respond by sending membership reports if they still have interest in those groups.

But be careful!To keep the traffic low, the response to a general query message must be done by only one host for a given group.

When a hosts receives a general query message, it delays the response: It sets a timer for each group to a different random value between 0 and 10 seconds.

If the host receives a response from another host, whose time for that group has expired earlier, the host cancels its response.

Note: Only one router on the LAN is designated for sending the query messages (the querier router).

Finally it is worth mentioning that IGMP messages are encapsulated inside IP datagrams.

To identify if an IP datagram is a multicast one, we must take a look at the PROTOCOL NUMBER field, if it is a 2, it means that it is using IGMP, multicasting.

MBONE (Multicast Backbone)

As the internet has been growing bigger and bigger, so have done applications that need to use multicasting. Multimedia and real-time applications have been the ones that needed to use it the most.

However, only a small fraction of the Internet routers support multicasting.

The solution is to use tunneling, as it provides a logical connection of multicast routers through non-multicast routers, called multicast backbone.

Unicast vs Multicast

To finish this article, let’s take a look at the main differences between unicast and multicast routing.

Unicast:

• Unicast routing is concerned about where the packet is going
• The destination IP address directly indicates where to forward the packet
• Forwarding is done hop-by-hop
• The routing table determines the interface and the next-hop router to forward the packet

Multicast:

• Multicast routing is concerned about where the packet came from
• The destination IP address (group) does not directly indicate where to forward the packet
• Receivers must be “connected” to the tree before the traffic begins to flow

Note that the multicast trees are spanning trees, where the source is located at the root of the tree, and the group members are the leaves of the tree.

The basic requirements that a group must fulfill to use multicast are the following:

• Every group member should receive only one copy of the multicast packet
• Non member must not receive a copy.
• A multicast packet must not be received by a router more than once (no loops)
• Paths from the source to each destination must be optimal (shortest path)

As you see, multicast is a technology with a HUGE amount of possibilities, almost every streaming service uses multicast nowadays.

I hope you liked this brief explanation about the topic.

In case you want to learn more about other Internet related topics, feel free to check my previous articles:

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps
• Chapter 4: DNS
• Chapter 5: DHCP
• Chapter 6: WWW (HTTP(S), HTML, Cookies…)
• Chapter 7: Firewalls & NAT
• Chapter 8: IP Tunnels

Finally, this series will reach to the 10th episode, so if you’ve liked my content please leave a clap, it makes a huge difference for me and keeps me motivated to write more articles!

You know the internet uses IP addresses to identify the devices where it has to send the packets and overall data. However, as time has passed, the number of available IP (version 4) addresses is almost exhausted and something had to be done to reuse or at least optimize the use of addresses.

Additionally, as more people learnt about the internet, and more and more files were uploaded to the net, serious threats appeared which had to be addressed.

In this article we’re gonna talk about how do our home devices (the ones connected to our home network, which is supposed to be secure) are connected to external networks such as the Internet (which are supposed to be “insecure”), and how do we protect our networks and devices from bad actors.

Firewalls

Usually routers distribute traffic according to a set of rules that only accounts for the destination address.

They do forward packets through a specific gateway (the default one, or another one that we must indicate) to the destination device.

But what if the traffic needs to be controlled according to other parameters?

For example imagine we are implementing the network of a certain company, and they ask for the following requirements:

• Equipment in the external network can only access the public web server.
• However, the marketing department can access the private web server, BUT not the public one.
• Traffic from one, and only one, specific device in the external network can reach the marketing network.
• Traffic going out the intranet network cannot reach the external network.

A possible solution would be to configure each single device to receive and send only the desired packets.

However this is not at all a scalable solution, and so another method had to be used. And this method are the FIREWALLS.

A firewall can be understood as a toll for control access.

It splits the network topology into an internal network, which should be secure and trusted, and an external network, which is somehow untrusted.

Goals of Firewalls

Firewalls are implemented generally for the following points, however they can have multiple other use cases.

• Single choke point for network traffic. This gives a solution to the scaling problem, and minimizes the risk.
• Access control rules. Only authorized traffic will be allowed to pass the firewall. This eases the administrative work.
• Secured Devices.

Types of Firewalls

1. Packet Filtering Firewalls

Packet filters inspect the IP packets reaching the Firewall and apply a set of rules to decide if they are forwarded or discarded.

• ACCEPT/FORWARD
• REJECT
• DROP

Filtering rules are based on information contained inside a packet, such as the network source/destination addresses, the transport source/destination addresses (TCP/UDP ports), the protocols used and the incoming/outgoing network interface.

Advantages:

• Simplicity
• Fast Evaluation
• Doesn’t change the traffic flow or characteristics

Weaknesses:

• Limited auditing
• Vulnerable to several (simple) attacks (spoofing attacks, source routing attacks, tiny fragment attacks…)

2. Stateful Filtering Firewalls

Also called Dynamic Packet Filtering, it takes into account the context, i.e. it maintains a history of previously seen packets to make better decisions about future packets.

It tracks open connections, maintains a table of them, and associates new connections requests with existing legitimate connections.

When tracking the open connections, the firewall maintains the following attributes (among others):

• IP addresses
• Ports
• Sequence numbers
• Connection Status

Advantages:

• More secure
• Defense against spoofing and DoS attacks
• Prevents TCP sequence-numbers attacks

Weaknesses:

• Rules are harder to write
• Still limited auditing capabilities

3. Proxy Firewalls

Proxy firewalls, or Application layer gateways (ALG) are devices that act as a relay for application-level traffic.

This means they can “understand” certain traffic application protocols. They are able to detect:

• If an unwanted protocol is attempting to bypass the firewall on an allowed port.
• If a protocol is being abused in any harmful way.
• If user credentials are enough to use some protocol.

Advantages:

• Even more secure
• Full auditing capabilities
• Hides the internal addressing scheme
• Harmful applications can be blocked

Weaknesses:

• Usually slower as they require much more resources
• A comprehensive knowledge of the protocol is needed
• Not available for all the application protocols
• Application protocols update frequently, and so the proxy has to be updated too
• Sometimes some extra client configuration is required

NAT

So now that we know how to protect ourselves from “external insecure networks”, we should know how to reach them.

But, you already told us that to reach a device we only need to know its IP address isn’t it? Yeah, that’s it.

However, you may have seen that MANY devices have really similar addresses between them, like the typical 192.168.1.1 (usually your home router).

This is because as the Internet grows bigger and bigger, the different networks which form the whole net had to find a way to differentiate between them.

There are PUBLIC networks and PRIVATE networks, being the first ones accessible for everyone, but the latest ones only for the ones who are inside that network.

Note how everyone can access directly the public addresses, but not the private ones.

For this exact reasons, almost ALL private networks use the same range of addresses (usually C class range, 192.168.0.0/16), as each and every private network is just accessible from its inside hosts, and they don’t need to have a different address from an unreachable host, as they are not connected from the first place.

Let’s put an example: We have 2 companies, A and B.

All the devices from network A can only access other devices from its internal network, but not the ones from B. This way, we can assign the same exact address to a device from network A and network B, as there will be no confusion between them.

That sounds legit, but what if now we need to connect from a certain device from network A to network B? Or maybe we want to send something from a private address to a public one.

We must know how to reach all this addresses.

This is when the NAT comes into play. It stands for Network Address Translation, and it is a mechanism that translates IP addresses (and port numbers), and allows to connect public and private networks, saving public IP addressing this way.

When we use NAT, two address translations are performed:

• One when the packet departs the NAT router.
• The reverse translation when the packet returns to the NAT router.

NAT Types

To do so, we use two main types of NAT.

Source NAT (SNAT):

• The router translates the source address after routing, just before the packet leaves the router.
• When response packets arrive, the router will perform the reverse operation (translation of destination address).

Destination NAT (DNAT):

• The router translates the destination address before routing, just when the packet arrives to the router.
• When response packets arrive, the router will perform the reverse operation (translation of source address).

Consequences of NAT usage

If IP addresses change, the checksums must be recalculated.

When a NAT is used, response packets should return to the same NAT router (to perform the reverse translation).

For this reason, in most cases there is only one NAT router to manage the Internet access.

NAT Limitations

For the above reasons, when using NAT, the Internet becomes a pseudo-connection-oriented network.

Furthermore, each NAT router becomes a bottleneck, as it must manage the translation of all ingoing and outgoing connections.

In case the router does also need to use a proxy firewall (ALG), it will have to do a whole bunch of tasks before sending each packet.

NAT conclusions

So forth, NAT has been extending the lifetime of IPv4, delaying IPv6 deployment. However, this is far from being an optimal solution, as there can be different behaviors for different NAT implementations, and new applications will need extra NAT support.

Firewalls & NAT in Linux (IP tables)

Alright, now you know the theoretical approach to how we connect to external devices, and how we protect ourselves from malicious actors.

Let’s take that into practice and see how we can implement NAT in Linux systems.

Netfilter

Netfilter is the packet filtering framework used in Linux platforms.

It contains chains of rules for the treatment of packets.

Each chain is associated with a different “role” of the host when processing packets.

Each network packet arriving at or leaving from the computer goes through at least one chain.

The five predefined chains are the following ones:

• PREROUTING: First chain that a packet finds when it comes from the network. It is located before the routing decision.
• INPUT: This chain applies when the packet is going to be locally delivered (the host is the destination), and it must perform some action in the host itself.
• FORWARD: the packet will enter this chain if it comes from the network, but the host is not the destination.
• OUTPUT: this chain applies when packets are sent by the host after having performed some action, and they need to be sent to another host.
• POSTROUTING: The packet will enter this chain after the routing decision has been made, and it will send the packet further.

Netfilter syntax

When a packet enters a chain, the kernel verifies if the rules inside the chain match this packet to know how to proceed.

Each rule contains some specifications to decide if the packets match it.

If a packet reaches the end of the chain without matching any rule, the default chain policy is applied (for example DROP).

How to configure the netfilter chains in the Linux Kernel? Using the iptables command.

Syntax:

iptables <table> <op> <chain> <pkt-match-condition> <action>

• table: Selects the table to work with

-t filter: selects packet filtering

-t nat: selects NAT table

• op: Rule operations within a chain

(-A) APPEND a new rule to the chain

(-I) INSERT a new rule in a position within a chain

(-D) DELETE a rule in a position within a chain

(-R) MOVE a rule from a position to another one

• chain: the chain name to operate with

Packet filtering: INPUT, OUTPUT and FORWARD

NAT: PREROUTING, POSTROUTING and OUTPUT

• Packet Matching Condition: The set of conditions that a packet must satisfy. There are 3 types of conditions:

Physical/link layer conditions: network interface where the packet is received/transmitted.

Network layer conditions: fields in the IP header

Transport layer conditions: fields in the transport header

• Action: The verdict is applied to the packet if all the rule conditions are satisfied.

Conditions:

• for the PHYSICAL/LINK layer:

-i, --in-interface [!] name 
-o, --out-interface [!] name

names of the interface via which packets will be received/sent

• for the NETWORK layer

-p, —protocol [!] protocol The protocol of the packet to check. It can be tcp, udp, icmp, or all

-s, —source [!] address[/mask] Source specification (network name, hostname, @IP…)

-d, --destination [!] address [/mask] Destination specification.

• for ICMP (if --protocol icmp is specified in the conditions of the network layer)

--icmp-type [!] typename This allows specification for the ICMP type

• for TRANSPORT layer (if --protocol tcp is specified in the conditions of the network layer)

--source-port [!] port[:port]

--destination-port [!] port[:port]

--tcp-flags [!] mask comp

As you can see there are many options available for the <condition> field, as the above ones are not all of them yet, but the main ones.

Examples

Finally I’ll show you some example of the iptables command and its use:

• Adding a rule in the INPUT chain to drop packets whose source IP address is 192.168.1.1 and transport protocol is TCP:

iptables -t filter -A INPUT -s 192.168.1.1 -p tcp -j DROP

• Adding a rule in FORWARD to drop packets whose source IP addresses are in range 192.168.1.0/24, the protocol ICMP and icmp type echo-request, and received in the eth0 network interface:

iptables -t filter -A FORWARD -i eth0 -s 192.168.1.0/24 -p ICMP --icmp-type echo-request -j DROP

• Adding a rule in OUTPUT to avoid outgoing http connections towards the www.google.com server:

iptables -t filter -A OUTPUT -d www.google.com -p tcp --dport 80 --syn -j DROP

• Adding a rule to do SNAT for all packets that leave the router using the eth1 interface, translating to the IP address 172.16.1.1:

iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 172.16.1.1

I know at first the rules used in our computers to filter the traffic can be a little hard to grasp, however once you have an overall understanding of how the whole Internet works, everything becomes much more clear.

In case you want to learn more about this topics, feel free to check my previous articles:

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps
• Chapter 4: DNS
• Chapter 5: DHCP
• Chapter 6: WWW (HTTP(S), HTML, Cookies…)

Finally, this series will reach to the 10th episode, so if you’ve liked my content please leave a clap, it makes a huge difference for me and keeps me motivated to write more articles!

In previous articles we talked about the different protocols used to configure the net and devices, how we protect from malicious behaviors, how to identify devices inside a network and route packets between them, and how we use browsers to access different files from the Internet.

However, we’ve been talking about public and private networks all this time, and we still don’t know how to connect between them. We know how to translate addresses using NAT, but this new addresses should be able to connect to other private networks, or at least find an “intermediary” network to route the packets through.

The tunneling solution

A tunnel is essentially a network communications channel between two networks.

Tunnels are used to transport another network protocol by encapsulating its packets.

They can be seen as Source Routes that circumvent conventional routing mechanisms.

If the tunnel is built at IP layer it is named IP tunnel.

To implement them, some routers must have two IP addresses to identify themselves, a public (outer) and a private (inner) one.

The outer router IP addresses identify the “endpoints” of the tunnel.

The image above represents a certain type of tunnel (GRE), which we will talk about it later, but just for understanding the general picture, take a look at the different IP addresses that appear in it.

We can see first of all two Class C addresses that identify the router inside each one of their private networks (192.168.1.0/24 for R1 and 192.168.2.0/24 for R2), and another IP address that identifies them in the public network, Internet (1.1.1.1/30 for R1 and 2.2.2.2/30 for R2).

Additionally note that we do also have 2 more IP addresses. These correspond to the tunnel, where we want to send the packets through.

When we use an IP tunnel, we may want to send other types of datagrams through that tunnel, and that’s why we encapsulate packets inside another packet, whose format is native to the public network.

There are several encapsulation protocols, to accomplish this.

• IP as encapsulation protocol itself (ipencap, 6in4, 4in6…)
• Other encapsulation protocols (SSH, TLS, IPsec…)

IP encapsulation protocol

Here we will analyze the first encapsulation protocol, as the other ones are quite different and made for a specific purpose.

To use the IP encapsulation method, each router creates a new network pseudo-device, which is associated to the outer IP address (take a look at the the “tunnel addresses” we mentioned with the previous image).

When a packet is sent through this pseudo-device, it is encapsulated on a IP packet and sent to the other remote border router.

Then, when a packet is received through this pseudo-device, the payload is extracted and treated as an incoming real IP packet.

Note: This network pseudo-device ONLY accepts IP packets from the other border router.

When we want to use the IP encapsulation method, we must include a new entry to the routing table to indicate which packets will be sent through the IP tunnel.

P.ex:

Destination		Gateway		Genmask			Iface 
...  ... 
192.168.2.0		0.0.0.0		255.255.255.0	tunnel0 
...  ...

This entry would tell the router that any packet towards 192.168.2.0/24 will be sent through the tunnel0 network pseudo-device, encapsulated in another IP packet, and sent to the other remote border router.

The outer IP packet

• Outer IP Header → Includes:
• IP source and destination addresses (“endpoints” of the tunnel)
• IP protocol field: type of encapsulation used
• Tunnel Header: some encapsulation types define a specific tunnel header to introduce extra parameters.
• The encapsulated inner IP packet

Let’s see some of the encapsulation types:

• IP in IP encapsulation

(does not have tunnel headers)

• GRE (Generic Routing Encapsulation)

Inside the IP payload we find the Inner IP Header and the Inner IP Payload.

The “Delivery Header” includes the Outer IP Header and the Tunnel Headers.

• IPsec (Internet Protocol Security)

This one is an encapsulation method worth mentioning because it can provide authentication by using Authentication Headers (AH), encryption by using Encapsulating Security Payloads (ESP), and finally mutual authentication at the beginning of the session and negotiation of cryptographic keys to be used during the session (Security Associations, SA).

Configuration of Tunnels

To configure tunnels in Linux systems, we will use the command ip tunnel.

Command options:

ip tunnel show - lists tunnels

ip tunnel add - add a new tunnel

ip tunnel change - change an existing tunnel

ip tunnel delete - destroy a tunnel

name NAME - select the tunnel device NAME

mode MODE - select the tunnel mode (modes for ipv4: ipip, sit, isatap and gre)

remote ADDRESS - set the remote endpoint of the tunnel

local ADDRESS - set the fixed local address for tunneled packets. It must be an address on another interface of the local host.

ttl N - set a fixed TTL “N” on tunneled packets. N must be in the range 1-255. 0 means that the packets inherit the TTL value, which is the default value.

dev NAME - bind the tunnel to the device NAME so that tunneled packets will only be routed via this device and will not be able to escape to another device when the route to endpoints changes.

nopmtudisc - disable Path MTU Discovery on this tunnel. It is enabled by default. Note that a fixed TTL is incompatible with this option (tunneling with a fixed ttl always makes pmtu discovery).

Despite being one of the shortest articles of the series, understanding how tunnels work on the Internet is a really powerful tool. I hope you got at least a first glimpse on what they are, and this article encourages you to do more research about the topic, as it is continuously being developed, and a lot more information can be found out there about tunneling.

In case you want to learn more about this topics, feel free to check my previous articles:

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps
• Chapter 4: DNS
• Chapter 5: DHCP
• Chapter 6: WWW (HTTP(S), HTML, Cookies…)

Finally, this series will reach to the 10th episode, so if you’ve liked my content please leave a clap, it makes a huge difference for me and keeps me motivated to write more articles!

In this chapter you’ll find an introduction to the protocols & languages used in the World Wide Web.

Content:

• History
• HTML
• HTTP Motivation
• URL/URI
• HTTP 1.0
• Cookies
• HTTP Proxies
• Dynamic Web
• HTTP 1.1
• Related RFCs
• Practical HTTP with nginx
• HTTPS (Secure HTTP)

History

Tim Berners-Lee is credited with having created the initial World Wide Web (WWW) during 1985–1991, while he was a researcher at the European High-Energy Particle Physics lab at CERN (Centre Européen de Recherche Nucléaire).

In this context, a multi-platform tool was needed to enable sharing documents between physicists and other researchers in the high energy physics community.

Tim Berners-Lee wrote a proposal that was a solution for enabling such

collaboration.

Four basic technologies were part of his proposal:

• HTML (HyperText Markup Language): a language to write documents.
• HTTP (HyperText Transfer Protocol): a protocol to transmit resources (like HTML documents).
• A WEB server: a software that serves resources like HTML documents.
• A WEB browser: a software that acts as client to send requests and process responses for resources available on a WEB server (like HTML documents).

HTML

HTML (Hyper Text Markup Language) is not a programming language like C or Java, but it is a markup language. This means that it is used to describe how content (text, images, etc) should be displayed (normally in a web browser).

HTML files are just text files that you can edit with any editor. There are also available “HTML editors” specially designed for writing it.

Let’s analyze a HTML fragment:

<html> 
    <head> 
    <title> Hello World</title> 
    <meta http−equiv="content−type" content="text/html; charset=UTF−8"> 
    </head> 
    <body> 
        Hello <b>World</b>!!!!!!! 
    </body> 
</html>

As you can see, HTML is pure text. However some of this text is considered hypertext which is the text enclosed between ‘<’ and ‘>’.

This hypertexts are called HTML tags, and they tell the browser to do something special with that text.

Some HTML tags have an opening tag and an ending tag. This is marked as <tag> ... </tag> .

Other tags however, are just composed of a single tag.

The HTML document is delimited by <html> and </html>. In addition, the HTML

document is divided in two parts:

• <head>. This part is optional. If <head> exists, it can contain several labels like <title>, <meta>, etc.
• <body>. Inside the body is where the whole HTML document is specified. All text, images, etc. are contained between <body> and </body>.

On the other hand, we can also use tags to create hyperlinks to other resources (like other HTML documents).

This is a fundamental feature in HTML. The hyperlink tag is <a>... </a>.

Example from an HTML code:

<html> 
    <head> 
        <title> Hello World</title> 
        <meta http−equiv="content−type" content="text/html; charset=UTF−8"> 
    </head> 
    <body> 
        Hello <b>World</b>!!!!!!! 
        Go to <a href=docs/otherdoc.html> another document </a> 
    </body> 
</html>

You can see how it is displayed in the browser simply by copying and pasting the previous code into an HTML document and dragging it into your browser. Of course it will not link you to any other document, as the path points to a document which does not exist.

Final HTML remarks:

On the other hand, blank spaces and new lines are called ”whites”. You can add as many ”whites” as you like to make your HTML file easier to read but browsers display consecutive whites as a single space. If you need to create a paragraph, you have to use the labels <p> ... </p>. For paragraphs, the browser will adjust the text lines correctly based on the window width. If you really want to force a new line, you have to use the <br> tag.
HTML has many tags but with a few of these tags, we can have an idea about how HTML works.

Some other useful tags are:

• <i> </i> Sets text in italics.
• <tt> </tt> Sets text in teletype.
• <h1> </h1> Sets text in type “header 1”. You can use numbers of headers in descending order of importance (size):
• <h2>: <h2>This is h2 </h2> . . . <h6>: <h6>This is h6 </h6>
• <hr> Prints an horizontal line.
• <center> </center> Centers text and images.
• <blockquote> </blockquote> Indents text.
• <pre> </pre> Pre-formatted text, i.e. spaces and line breaks between these tags are maintained.
• <!-- text comments... --> Comments in the HTML file.
• and many more tags…

HTTP Motivation

Initially, HTTP (Hypertext Transfer Protocol) arised from the necessity of creating hyperlinks in HTML documents to resources that are not on the same host.

HTTP is a text protocol and it is based on a client/server model that can be used over a TCP/IP network to deliver virtually any resource of the World Wide Web (WWW). For now, we will consider that a resource is just an HTML document. An HTTP server or WEB server is a network daemon that uses by default the well-known TCP port 80.

HTTP clients, generically called WEB Browsers (e.g. firefox or lynx), send HTTP requests to the HTTP servers asking for a resource and the server responds with the requested resource.

URL / URI

The first issue to implement HTTP is to define how to identify resources.

The identifiers used in HTTP were initially defined by Tim Berners in 1991. They were called URLs (Uniform Resource Locators) and they were first used to allow authors of HTML documents to establish hyperlinks in the WWW.

An URL is just a text string with a standard format that allows you to name a resource based on its location on the WWW. In 1994, the URL concept was incorporated into a more general concept called URI (Uniform Resource Identifier).

URI is the standard name for resource identifiers in the Internet, but the term URL is still widely used. The simplest URL/URI format is as follows:

protocol :// hostname / directory / resource

But other information can also be present in the URL:

protocol :// username : password@hostname : port / directory / resource

Important note: If in the URL there is not any resource (filename) specified, it is assumed that the client is asking for a file called index.html or index.htm.

As its name suggests, this file contains an HTML file with the website index.

On the other hand, we can use absolute or relative paths in HTTP hyperlinks. In an HTTP server, absolute paths are related to a directory called DocumentRoot.

This parameter is defined in the configuration file of the HTTP server.

For example, a typical DocumentRoot when using Linux is /var/www. In this case, the URL http://www.example.com/images/upc1.gif refers to a file called upc1.gif that is stored in the HTTP server in the directory /var/www/images.

We can see the difference between absolute and relative paths in the following piece of code:

<html> 
    <head> 
        <title> Hello World</title> 
    </head> 
    <body> 
        <p>Hello <b>World</b>!!!!!!!</p> 
        <p>Go to <a href=docs/otherdoc.html> another document </a></p> 
        <p>You can visit the UPC home page at <a href="http://www.upc.edu">UPC home</a>. </p> 
        <img src="/images/upc1.gif"> 
        <img src="/images/upc2.gif"> 
        <img src="http://www.example.com/images/upc1.gif"> 
    </body> 
</html>

HTTP 1.0

As we already said, HTTP is a protocol that uses the client-server model.

To start using it, we do need the client to know the server port and include it in the URL.

How does it work?

First the HTTP client opens a TCP connection and sends an HTTP request to an HTTP server. If everything is correct, HTTP server returns a response which contains the requested resource.

Once the response is delivered, HTTP server closes the TCP connection.

Note: HTTP is a stateless protocol. It does not maintain information between different requests.

HTTP Requests

When we do have an HTTP request, the first line is the ONLY mandatory one. It contains:

• “Request method”
• Path to the resource
• HTTP version

Then it follows a blank line (to tell the protocol it’s the request end).

The minimal HTTP 1.0 request is something like the following:

GET / HTTP/1.0 
[blank line]

GET is the most commonly used request method and it means “give me the resource X”. After the GET we find a “/” and this means the resource we are requesting is the index file of the WEB server.

Another example requesting another file would be like this:

GET /images/upc1.gif HTTP/1.0 
[blank line]

In this case the client is requesting a file called upc1.gif that is stored in the HTTP server in the images directory.

Headers

Request & Responses can have header lines.

This Headers are text lines that provide additional information or functionality in requests/responses.

The usual format for Headers is: “Header-Name: value1, value2” , ending with CR+LF (blank line). The header name is NOT case-sensitive.

Examples of Headers (the both are equivalent) would be something like this:

Header1: some-long-value-1a, some-long-value-1b 
 
Header1: some-long-value-1a 
    some-long-value-1b

HTTP 1.0 defines 16 headers, though none is required. Typical headers included in the requests are:

• From (email adresses of the user who makes the request)
• User-Agent (name of the browser and OS)

An example of a request with headers would be the following:

GET /path/file.html  HTTP/1.0 
From: user@example.net 
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:24.0) Gecko/20100101 Firefox/24.0 
[blank line]

HTTP Responses

HTTP Responses are also composed of text lines. The first text line of an HTTP response is the status.

Typical status lines are:

HTTP/1.0 200 OK 
HTTP/1.0 404 Not Found

Let’s explain what do the numbers mean:

The first digit identifies the general category of the status:

• 1xx indicates and informational message only
• 2xx indicates success of some kind
• 3xx redirects the client to another URL
• 4xx indicates an error in the client side
• 5xx indicates an error in the server

Examples

• 200 OK
• 301 Moved Permanently
• 302 Moved Temporarily
• 303 See Other
• 404 Not Found
• 500 Server Error
• 503 Not Available

Furthermore we can also add Headers to the responses. The headers typically included in responses are:

• Server (identifies the server software)
• Date
• Last-Modified (date of last modification of the resource being returned) ← Used for caching

Final HTTP 1.0 example

Imagine we want to retrieve the file http://www.example.com/path/file.html using HTTP 1.0, the first step is to open a TCP connection with the server using the default port 80.

Then through this connection, the client can send a request like the following:

GET /path/file.html HTTP/1.0 
From: user@example.net 
[blank line]

And the server would respond with something like the following:

HTTP/1.0 200 OK 
Date: Mon, 21 Oct 2013 22:29:59 GMT 
Content-Type: text/html 
Content-Length: 50 
[blank line] 
<html> 
    <body> 
        <h1>It works!</h1> 
    </body> 
</html>

Once the response is received, the client closes the TCP socket.

Cookies

As we explained previously, HTTP is a stateless protocol.

A cookie is a piece of information (UTF8 text) sent from an HTTP server and that is stored by the browser in the client’s filesystem.

Sometimes cookies are also called “footprints”.

These cookies provide a state (memory of previous events) into otherwise stateless HTTP transactions.

Without cookies, each retrieval of a webpage, or even a single component, is an isolated event.

The most common uses of cookies are:

• User Control (For ex: when a user enters his username and pwd, a cookie can store this information so there is no need to enter them again in a later visit to the web server).
• Getting information about user’s browsing habits.

How do cookies work?

The HTTP server sends lines with the Set-Cookie header if the server wishes the browser to store these cookies.

Set-Cookie is a directive for the browser to store the cookie and send it back in future requests to the server (subject to expiration time or other cookie attributes).

Let’s see a visual example:

We can describe the process going on in the above picture:

1. The client sends a regular request.
2. The server asks the client to store that cookie.
3. The client sends the cookie in a subsequent request, when it is needed.

It is worth knowing that the cookies do have more fields, like path and domain, which help in deciding when to send it or not.

As you can imagine, cookies can cause (and are actually causing) privacy problems.

HTTP Proxies

An HTTP proxy is a program that acts as an intermediary between a browser and a Web Server.

HTTP proxies are commonly used for security (a single point of control) and efficiency (caching).

There are two basic types of proxies (from users point of view):

• Tranparent: A transparent proxy intercepts normal communication at the network layer without requiring any special client configuration. Clients do not need to be aware of this proxy existence.
• Non Tranparent: This types of proxies receive requests from the clients and sends the requests to the servers. The responses go the same way back also using the proxy. Therefore, a proxy must have functions of either client & server. A non-tranparent proxy can use another transparent or non-transparent proxy to reach the final server.

Clients send their requests to the proxy instead of the real server specified in the URL. ← The proxy IP address and port is defined in the browser

HTTP requests using a non-transparent proxy must include the full URL of the resource (not only relative path), as the proxy has to know which server it must send the HTTP request.

A request using a non-transparent proxy would be something like this:

GET http://www.somehost.com/path/file.html HTTP/1.0 
[blank line]

Lastly, it is nice to know there are many Open Source Proxies used widely.

Dynamic Web

In today’s Web, the content is not static, but documents are generated on the fly by servers with information provided by clients.

As a result, WWW is not a huge database of documents, but a platform to implement services & applications.

Common applications of the dynamic web are searching engines, remote access to corporate applications and databases, etc.

CGIs

Note: We do have several ways of implementing the dynamic Web. In this document, we will only explain CGIs, as they are easier to understand, and they were the first method used for that specific purpose.

CGIs or Common Gateway Interfaces are a standard procedure through which HTTP servers can use external applications to dynamically generate content.

When we want to use a CGI, in the URL we must identify:

• An executable program (the CGI itself)
• The parameters with which the CGI has to be executed.

However, when we use CGIs we find some issues we need to solve.

The first one is how a web server knows that is has to execute a program instead of sending a resource. → A usual solution is to store all the CGIs in a special directory, typically called /cgi-bin/. In this way, if a client asks for www.examplecom/cgi-bin/program the server knows that it must execute program instead of sending it.

The second is how to send the parameters to the program. When we use GET, the parameters are encoded in the URL. These parameters are added to the URL after a “?” character, and separated by the character “&”.

Example:

http://www.example.com/cgi-bin/program?param1=value1&param2=value2...

Note: Spaces are translated into the ‘+’ character, and ASCII characters can also be sent in the %NNN format.

Finally, before executing the CGI, the Web server establishes a special context for the program using environment variables, to ensure a correct execution of it.

These variables are:

CONTENT_LENGTH, CONTENT_TYPE, REMOTE_HOST, REMOTE_USER, REQUEST_METHOD, SERVER_NAME, QUERY_STRING, GATEWAY_INTERFACE, HTTP_’VERSION’

For GET requests, the QUERY_STRING variable takes the value of the parameters and the CGI can use them as the client has specified.

For the response, the CGI writes it to the STDOUT. Then the server reads this answer and sends it to the client through the socket.

Depending on the type of web server, the CGI application can act in two different ways:

• NPH Server (No Parse Headers) → The CGI writes the complete response including the HTTP headers.
• PH Servers (Parse Headers) → The CGI writes the complete response without the HTTP headers, and it must pass information on how to form them.

The most common web servers are NPH.

Finally, it should be remarked that CGIs are not the most efficient solution because a process is created per request, and nowadays there are too many requests to be handled in this way.

Today’s more efficient solutions to create dynamic websites are languages like Javascript, Python, PHP, Java servlets, etc.

Other (common) methods in Requests: HEAD and POST

• The HEAD method is used when we want a response only with the status line and headers (without a body).

HEAD is useful when the resources from the server are not actually needed. This can be the case in which we need to make some tests but we do not want to download a resource (which can be heavy)

• The POST request is used for dynamic Web. The difference between POST and GET is that POST requests use the body of the request to send parameters instead of coding the parameters directly into the URL.

CGIs with POST use the STDIN to receive the request body (program’s parameters) instead of using the QUERYSTRING variable

HTML Forms

An HTML form allows a client to send parameters to a WEB server.

The tag to declare a form is <form> … </form> where different elements can be inserted inside, like text input elements, codes, images, checkboxes…

These elements are inserted into the form using the tag INPUT. All items of the form have a “type” attribute and they might have a “name” attribute.

There are 2 special elements in forms: RESET, which clears the form to its initial state, and SUBMIT, which presents a button to send the form to the server.

Example of an HTML form, and how it is displayed in the browser:

<html> 
    <head> 
        <title> Website title </title> 
    </head> 
    <body> 
        Form to select parametres to send to the server. 
        <form ACTION="/cgi−bin/process" METHOD="GET"> 
            Enter a name: <INPUT NAME="a" TYPE="text"> <br> 
            Enter a password: <INPUT TYPE="password" NAME="b" 			MAXLENGHT="8"> <br> 
            Checkbox: <INPUT TYPE="checkbox" NAME="c"> <br> 
            <INPUT TYPE="reset"> <INPUT TYPE="submit"> 
        <br> 
        </form> 
    </body> 
</html>

When the form is submitted, the client generates an HTTP request using the method showed in the METHOD attribute (GET in this case), to execute the script or application indicated in the ACTION attribute.

The most common question regarding Dynamic Web is: Should I use GET or POST?

Each method has its advantages and drawbacks.

GET

When using GET, the parameters are encoded in the URL. This can be useful in many cases, but if we want to protect some sensible information, the GET method could cause security vulnerabilities, because the parameters will be shown to everyone in the URL.

Another drawback of GET is that it does not allow to send binary files in the body of the request.

However, GET is useful to perform requests and store the results together with the associated URL.

GET also allows to use the back buttons to go to previous results.

POST

With POST, the parameters are sent in the body of the request, so the parameters are not visible in the browser as a query string.

In general, GET is useful for operations which always give the same results (idempotent).

POST means “carry out” an action with a “side effect” or a change of state (non-idempotent operations).

HTTP 1.1

Introduction

The main difference between protocols 1.0 and 1.1 is that on 1.1 it is compulsory to add a HOST header

The most important improvements made to the protocol are the following ones:

Host Header

Provides (a more) efficient use of IP addresses. With 1.1, multiple domains can be served from a single IP address.

For example, we could have www.example.com and www.example.net on the same server. Thus, HTTP1.1 must specify in the header to which host the request is destined. A minimal HTTP 1.1 request with the hostname would be something like:

GET / HTTP/1.1 
Host: www.example.com:80 
[blank line]

Where we can see the domain name (or IP) of the WEB server, and the port number (in this case 80, as it’s the default port for HTTP).

Chunked Encoding.

Allows a faster response for dynamically generated pages. Pages are divided and sent in chunks (fragments). This way, responses can be sent before its total content or length is known.

When we use HTTP 1.1, it does send the body of the message as several fragments, followed by a line with a “0”. Optionally followed by the page footer.

Each chunk consists of two parts:

1. A line with the size of the chunk in hexadecimal + CR + LF
2. Data + CR + LF

(CR + LF == blank line)

Example WITHOUT chunks:

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 42
[blank line]
abcdefghijklmnopqrstuvwxyz1234567890abcdef

Same example WITH CHUNKED DATA:

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked
[blank line]
1a
abcdefghijklmnopqrstuvwxyz
10
1234567890abcdef
0
[blank line]

Persistent Connections

A TCP connection is not opened/closed for each request. By allowing multiple HTTP transactions in every TCP connection, we can reduce the total transmission delay.

In HTTP 1.0, TCP connections are closed after each request/response by default.

As we know, opening/closing TCP connections requires a substantial amount of CPU time, bandwidth, and memory. In practice, most web pages consist of several files (linked HTML documents, images, etc.) that are located on the same server.

Consecutive requests (and their associated responses) can be more efficiently transmitted by allowing multiple requests/responses to be sent over a single connection.

In HTTP 1.1, persistent connections are used by default.

Once the client decides it doesn’t want to send any more requests, it sends a request with a Header which includes: “Connection: close”.

Then the server has to close the connection after the reply.

If the server is the one who sends the header with “Connection: close” then the client cannot send more requests, and it must close the connection after the response is received.

*Note that a server may close a connection before it answers all the requests.

The HTTP 1.1 client can also send multiple requests through a single connection without having received any response (pipelining).

On its side, an HTTP 1.1 server must store queued requests while it cannot process them, and it must send the responses in the same order as it received the requests. If a request includes the header ”Connection: close” , the server must interpret this as that the request is the latest and it must close after sending the corresponding response.

Finally, it is worth to mention that typically, clients (browsers) open several simultaneous persistent TCP connections with each server.

Caching

Headers to implement caching are provided with HTTP 1.1.

This allows a faster response and bandwidth saving.

All the responses (including errors) except the “Continue” ones (status 100) should include the header ”date“. This header provides a time stamp which is used by HTTP 1.1 to implement caching. These time stamps use the Greenwich Mean Time (GMT).

There are two headers called ”If-Modified-since“ and ”If-Unmodified-Since“ that can be included in HTTP requests.

Continue

When we want to send a message with a big body (big file), we can use the “Continue” (status 100) mechanism.

This mechanism is used to determine if we do really want to receive a message or not, as if we cannot process it, we can save time and resources by rejecting the message.

To use the Continue mechanism, clients must include the header: “Expect: 100-continue”. Then if the server is going to process the request must respond with 100 (Continue) status.

Note: A client should not send the Expect header if it’s not going to send any body in its request.

As a final remark, it is worth mentioning that

HTTP 1.1 clients should:

• Include host header in each request.

• Accept responses with chunked data.

• Accept persistent connections or include the header ”Connection: close”.

• Manage the response ”100 Continue”.

HTTP 1.1 servers should:

• Require the host header in requests.

• Accept absolute URLs.

• Accept ”chunked” requests.

• Manage persistent connections (or use the header “Connection: close”)

• Properly use the status ”100 Continue”.

• Include the date in the header ”date” in each response (except Continue).

• Manage requests with headers “If-Modified-Since” or “If-Unmodified-Since”

• At least, support the methods GET and HEAD.

• Support HTTP 1.0 requests.

NGINX

Introduction

NGINX (pronounced “engine x”) was born as a reverse proxy with minimal HTTP WEB server functions.

It addresses most of the server activities performed in the web world, such as:

• Serving static content (i.e. static web pages, files, etc.).
• Dynamic content provisioning (e.g. using FastCGI)
• Caching content to speed up service provisioning.
• Load balancing for any TCP or UDP service (e.g. HTTP requests, DNS resolution requests, etc.).
• Securing channels for applications not supporting HTTPS connections.

NGINX can run in several platforms, including Linux, FreeBSD, Windows and MacOS. Without going into the technical details, developers and network engineers tend to prefer nginx over Apache as:

• It’s easier to configure.
• It scales better in terms of resources and performance as the load increases.
• It is extremely flexible: provides load balancing, cache, web server, etc. in a single piece of software.

Debian-based distros store the nginx configuration files in the directory /etc/nginx.

NOTE: if you change the configuration of the daemon you have to stop and start it to apply the changes.

To stop nginx type:

# /etc/init.d/nginx stop

To start the daemon:

# /etc/init.d/nginx start

Virtual Hosts

A virtual Host is a web site served by the HTTP server. Each virtual host has it¡s on configuration file, which can be found at:

/etc/nginx/conf.d folder.

Apart we can find a list of available virtual hosts at /etc/nginx/sites-available

The default virtual host is (unless we modify it) the first configuration file, but we can also specify a virtual host by including the default_server keyword next to the listening port inside its configuration file.

CGIs with nginx

CGIs are not directly supported by nginx, but we can use the FastCGI library to provide the CGIs functionality along with nginx.

Load Balancing

One of the most useful features of the nginx server is the capability to balance the load.

We can achieve load balancing just by creating and upstream configuration as:

upstream tcgi−app { 
    server www1.example.com; 
    server www2.example.com; 
} 
server { 
    listen 80; 
    server_name www.example.com ; 
    location / { 
        proxy_set_header X −Real−IP $remote_addr ; 
        proxy_set_header X −Forwarded−For $remote_addr ; 
        proxy_set_header Host $host ; 
        proxy_pass http://tcgi-app$request_uri; 
    } 
}

The nginx server provides several approaches to balance the traffic:

1. Round Robin: The default approach
2. Least busy: NGINX will try not to overload a busy server with excessive requests, distributing the new request to a less busy server instead.
3. Session Persistance: The clients are tied to a particular application server per session.
4. Weights: It is also possible to assign a certain weight to a server. This is interesting when using the Round Robin approach, and we can use the weight parameter next to the server configuration to weight the traffic addressed to that server.

HTTPS

Up to this point, we’ve been seeing how does the HTTP protocol work, but we’ve encountered many times the same problem… Security.

To solve this issue, a new protocol had to be implemented, and as you may have guessed, the protocol was HTTPS.

To implement this security layer on top of HTTP, different cryptographic methods have been used.

The main algorithms used to implement HTTPS are the following ones:

1. SYMMETRIC Cryptography

What does that even mean? It just means that either the sender and the receiver use the same key both to encrypt and decrypt the message.

It does NOT mean that same algorithms are used, nor that the messages are encrypted in a symmetric way, simply that the same key is shared.

That’s why it’s also called Symmetric key cryptography, or shared secret cryptography.

The most used algorithm en symmetric cryptography has been the DES (and triple DES), until the AES was invented, which can have longer keys than DES.

Using this cryptographic systems we won confidentiality in our message sharing process through the internet, but another problem is detected when using symmetric cryptography, and it is the fact of sharing keys between participants, as they should know each other and share they key between them before initiating the data transmission.

2. Asymmetric Cryptography

Brought by the problem that we just described, another type of Cryptography appeared, where sender and receiver do NOT need to share any secret nor key. This is called Public Key Cryptography.

In this type of cryptography, each user has a pair of keys:

• A public key (everyone can see it)
• A private key (only the user knows it)
• One of them will be used to encrypt the message (public), and the other one to decrypt it (private).

Asymmetric algorithm example
Alice wants to send an encrypted message to Bob. To do it using asymmetric cryptography she does the following:

• Alice uses Bobs Public key to encrypt her message.
• The message is encrypted thanks to Bobs Public key, and it is send to Bob (now only Bob’s key can decrypt that message, no one else, not even Alice)
• Bob receives the encrypted message and he uses its Private key to decrypt it, and this way he can read the message.
• The most used algorithms in Asymmetric Cryptography are RSA, DSA (Digital Signature), ECDSA…

3. HASH Function

The Hash function appeared due to the performance problem we had when searching for data inside DataBases, which were usually alphabetically sorted.

It was seen that using the hash function was MUCH more efficient as:

• A number is assigned to each message’s character.
• All this numbers are added.
• The 100 module is applied to the result of the previous step.
• We obtain the HASH value.

This HASH value is much easier to fins inside a DB.

One of the main characteristics of the Hash function is that it doesn’t matter the variable inputs that it gets, but the output is always fixed.

A problem that this may cause is that multiple inputs can produce the same output, and so collisions appear.

So how can we reduce the collision probability?

1. Increment the module operation (mod10000)
2. Use a different codification (p.ex. ASCII)

But is there a way to achieve a virtually impossible collision probability?

— > Using the CRYPTOGRAPHIC HASH

This algorithm uses SHA256, which works like this:

• We introduce a variable Input
• Returns 256 bits (output)

The difference is that now is practically impossible to guess which is the original message only from the output (only with brute force attacks).

So as we cannot return to the original message, it is possible that collisions appear, but we won’t know where this collisions come from.

4. Digital Signatures

Digital Signatures generation:

1. Alice calculates the message hash (HASH FUNCTION)
2. Alice encrypts the hash with her private key (RSA)
3. Alice sends the message and the signed hash to Bob.

(Note: The 256 bit hash identifies univocally the message, and the private key identifies Alice univocally)

1. Bob receives the encrypted message → He calculates the message’s hash.
2. He decrypts the signed hash using Alice’s public key.
3. If both keys coincide, the signature is valid, and Bob can access the message.

5. Hybrid Cryptography

We can divide the message that we want to send in blocs, encrypting each block with the public key from the destiny.

Asymmetric cryptography is more secure, but Symmetric cryptography is faster.

With a combination of them both we can achieve much better performance.

To use them together we can do it like this:

• Asymmetric Cryptography is used to exchange a session key
• Then Symmetric Cryptography is used to share the encrypted data.

6. Digital Certificates

The Digital Certificates contain:

• Camps to describe the identity
• Serial Number
• Validation Period
• The Public Key algorithm
• The CA Public Key
• The algorithm used to sign the certificate
• Camps with the certificate’s purpose and additional information

7. OpenSSL

OpenSSL is the most used implementation to create and manage keys and certificates

8. HTTPS

Finally, after doing a quick cryptography introduction we can get back to HTTPS.

This protocol is implemented with the security that our data cannot be intercepted while it is traveling through the internet.

WebApps use Hybrid Cryptography and Digital Certificates to secure their transactions.

To secure the transactions, the browser does various security checks to make sure that we are connected to the correct server:

1. Is the Digital Certificate well signed?
2. Do I know the CA public key?
3. Has the certificate expired?
4. Is the URL the same as the certificate’s one?

Note: Browsers have lists of Public Keys from CAs, to be able to comply with point 2.

TLS Tunnel:

To prove that we are connected to the correct server the following steps need to be checked:

From the Browser

• A Symmetric Session Key is generated (Ks) and sent to the server.
• The Session Key is encrypted with the Public key that appears on the certificate.

From an HONEST Server

• It will have the corresponding Private Key (Kp) to our Public Key (Ku) (if the server was not the correct one, it would not have the Private Key).
• The message is decrypted using the corresponding Private Key, and we send the Session Key back to the browser to demonstrate that we are the correct server, and that we have the Private Key that is needed.

And this would be an introduction on how does HTTPS work, and the methods used to secure or communications online.

This is not at all everything that the HTTP protocol involves, as other versions like HTTP 2.0 and 3.0 exist, but we’ll not cover them here.

However, I hope this article provides a solid basis to understand an overall picture on this protocol, and if you still have any doubts, hit me up at akakush19@protonmail.com, I’ll be happy to share thoughts with you!

If you want to check my previous articles to understand some other Internet concepts feel free to do so!

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps
• Chapter 4: DNS
• Chapter 5: DHCP

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

In the last article, we talked about DNS, how does the internet know where each host is allocated, and translates each host name to its IP address.

However, having a static IP address for each host is not at all scalable.

That’s why nowadays we use DHCP. This protocol automates the process of configuring the network parameters of network devices.

This means that each time a device opens, and connects itself to the Internet, it uses the DHCP protocol to configure its parameters, and to be able to be properly connected to the rest of the world (public) devices.

These parameters include the assigned IP, the network mask, the IP address of one or more DNS servers and the IP address of the default router.

DHCP consists of two components:

• A mechanism for allocation of network addresses (IPs) to devices.
• A protocol for delivering network addresses and other host-specific configuration parameters from the DHCP server to the device being configured.

DHCP Allocation Mechanisms

DHCP supports 3 mechanisms for IP address allocation:

1. Manual Allocation: A particular IP address is pre-allocated to a network device by an administrator. The network device is typically identified by its MAC address (but other methods like secret keys can be used).
2. Automatic Allocation: DHCP automatically assigns an IP address permanently to a device, selecting it from a pool of available addresses.
3. Dynamic Allocation: DHCP automatically assigns an IP address to a client for a limited period of time (or until the client explicitly relinquishes the address).

The 3d mechanism (dynamic) is by far the most widely used, as it is the most scalable of them all.

It allows automatic reuse of addresses, as the huge majority of internet hosts will just connect temporarily, or they won’t need a static IP address.

The period of time over which a network address is allocated to a client is called the lease.

The dynamic allocation mechanism works as follows:

• A client requests the use of an address for X period of time (lease time).
• The allocation mechanism guarantees that the assigned address will not be reallocated to any other device during the requested time.
• Later, the client may extend its lease with subsequent requests.
• Each time the client requests an address, it usually includes the previous assigned address and the allocation mechanism attempts to assign the same network address.
• If the client does not need the address anymore, it can issue a message to release the address back to the server.
• Finally, the client may ask for a permanent assignment by asking for an infinite lease.

Obviously the DHCP server needs a database or persistent storage of network parameters for network clients, as it has to know the state of the configured hosts to operate properly.

DHCP servers usually store a key-value entry for each client, which is a unique identifier, and contains the configuration parameters for the client.

DHCP Protocol

DHCP messages use ports 67 and 68.

The messages that are sent from client → server use the server port 67.

The messages that are sent from server → client use the client port 68.

Process to allocate network addresses:

1. The client broadcasts a DHCPDISCOVER message from the port 68, to the port 67 of the server.
2. Then, each DHCP server available in the network may respond from its port 67 to the port 68 of the client with a DHCPOFFER message, which includes an available network address and other configuration parameters.
3. Then the client selects a server, and broadcasts a DHCPREQUEST message to request a specific IP address. It does also do this from its own port 68, to the port 67 (still broadcasting).
4. The server responds with a DHCPACK message with the requested parameters, from its port 67 to the port 68 of the client, who will now have an assigned IP with a lease time.

There are other DHCP messages that servers and clients can use, such as DHCPNAK (server to client indicating network address is incorrect), DHCPDECLINE (client to server indicating network adress is already in use), DHCPRELEASE (client to server relinquishing network address and cancelling remaining lease) and DHCPINFORM (client to server asking only for local configuration parameters).

Using DHCP in Linux

DCHP server (dhcpd)

You can start, stop, get the status or restart the DHCP server with:

server# /etc/init.d/dhcp3-server start/stop/status/restart

DHCP Client (dhclient)

You can start/stop the DHCP client for Linux with the command:

client# dhclient3

The previous command sends a DHCP discover through all the interfaces of the host.

However, it is typical to start the client only in some interfaces of your network, for example:

client# dhclient3 eth1

And now we would only be looking for dhcp servers through eth1 interface.

If you want to release a certain interface (eth1 in this case) lease:

client# dhclient3 -r eth1

When the DHCP client is started, it reads a configuration file to know what to do, which is typically located in /etc/dhcp3/dhclient.conf.

DHCP Commands Summary

Server

• Script to start/stop the DHCP server: /etc/init.d/dhcp3-server
• Configuration file for ISC DHCP server: /etc/dhcp3/dhcpd.conf
• Current leases of the DHCP server: /var/lib/dhcp3/dhcpd.leases

Client

• Start, stop, restore the DHCP client: dhclient3
• Configuration file for the DHCP client: /etc/dhcp3/dhclient.conf
• Current leases of the DHCP client: /var/lib/dhcp3/dhclient.leases

Other important commands:

/var/log/syslog

This is the file where is stored the DHCP event log.

This is reaching to an end, DHCP itself is not the most complex protocol used in the Internet, but it is by far one of the most used of them all.

I hope this article helped in clarifying some ideas, and if you still have any doubts, hit me up at akakush19@protonmail.com, I’ll be happy to share thoughts with you!

If you want to check my previous articles to understand some other Internet concepts feel free to do so!

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps
• Chapter 4: DNS

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

When we want to send data through the internet, we must know the IP address of the destination device to send an IP datagram.

However, IP addresses (IPv4) are something like 192.168.0.1, or 172.16.255.255, and for humans those numbers are hard to remember. We tend to remember easier names than numbers, and that’s why the DNS was invented.

DNS stands for Domain Name System, and it links names with IP addresses in the Internet.

How does it work?

Now, the question is how to implement the DNS. How can our computer know which IP corresponds to each name?

The first time the DNS was implemented, the Internet was so small that simply having a local file of translations in each computer was enough.

In Unix-like systems, such file is the /etc/hosts, and it contains lines of text with name translations in the form “address”, “long name” and “short name”.

If we take a look at our file:

$ cat /etc/hosts                                        
## 
# Host Database 
# 
# localhost is used to configure the loopback interface 
# when the system is booting.  Do not change this entry. 
## 
127.0.0.1	localhost 
255.255.255.255	broadcasthost 
::1             localhost

We can see our localhost and broadcast addresses.

However, the Internet continued evolving, and it became so huge so quickly that another system had to be thought.

Take a look at the figure below, note that it is displayed logarithmically

The first proposed solution was to store all the translations in a server and only use the “hosts” file for specific local translations, that’s why we can only see my localhost and broadcast address.

The file and the centralized server were maintained by an organization called InterNIC.

Obviously they realized that this wasn’t either a good idea, and that a more decentralized approach should be clearly taken, as more and more users joined the network.

Decentralized Solution

To provide efficient name translations and delegation is best to use a distributed database (multiple servers) with a variable depth hierarchical name space.

This literally means that the names can have different lengths and that they are ordered in a hierarchy.

This hierarchy allows name delegation and name uniqueness. With this solution we can have several devices with the same short name or unqualified name (such as www) but each device can have a different long name or Fully Qualified Domain Name (FQDN), such as sales.microsoft.com.

Domains

The name space of the DNS tree is divided into domains. A domain includes all the names ending with that domain suffix. For example, the domain .com includes all the names ending with .com, like medium.com, github.com, and a long long list of names.

Domains are classified according to their level or depth inside the DNS hierarchy.

First level domains are managed by governmental organizations, countries or special agencies related with Internet.

Second level domains are managed by private entities. An example of second level domain is medium.com.

Additionally, there is a top level or root domain. This domain is the dot domain: “.”, and it is managed by InterNIC.

Take a look at the following picture

If we count the number of domains in this picture we can see there are 7: “dot”, .com, example.com, .net, example.net, left.example.net, right.example.net.

Each domain has one or more network devices connected to it. We will refer to this network devices as the “tree leaves”.

We can see how the domain .com has 2 leaves: bob.com and nsc.com.

Zones

When we want to delegate petitions of name translations, the DNS tree is administratively divided into zones. A zone is an administration point, that contains a configuration file containing a set of translations that is managed by a master or primary name server.

Each zone can have multiple servers, and a server can serve multiple zones, but to make it easier to understand, we are going to assume that each zone is implemented with just one server and that each server just serves that zone.

Take a look at the possible zones that we could have with the previous tree:

As you can see we’ve separated the 7 domains into 6 zones. The server root is in charge of the zone dot (“.”), and it has translations of all the names except the ones that it has delegated. In our case, the root server delegates .com and .net, so in case there are no more devices it will just know the translation of nsc and nsn to delegate all the names ending with .com and .net.

The domain .com is managed by nsc and the .net by nsn.

The server nsc manages all the names ending with .com except those ending with example.com.

The domain example.com has been delegated to a server called nsce, thus the translation of bob.com is stored in nsc but the translation of alice.example.com is stored in nsce.

In the nsn branch we could see similar zones except the left one, which we chose to make a single zone for either example.net and left.example.net. Why did we choose to do that? Because there is only one device that hangs from the left branch, and it can be already managed by nsne, there is no need to add another server there to store the information of bob.left.example.com. Of course we could have divided that into two different zones, one for example.net and another one for left.example.net, but they both would have been served by nsne.

The network addressing is done orthogonally respect to the hierarchy of the domain.

In our example, we have a single IP network 10.0.0.0/24, which is addressed as follows:

Obviously in the “real” Internet we have many different IP networks connected with routers.

Implementation with RRs

The DNS database in each server is implemented with Resource Records (RRs). RRs are text lines that define the configuration of the DNS tree nodes.

The general format of the RR is the following:

Owner	[TTL]	Class	Type	RDATA

• Owner: RR owner, a name.
• TTL: the time that a RR may be cached by any resolver (optional).
• Class: Resource records belong to a class. Typically the class is IN (Internet)
• Type: The RR type.
• RDATA: Record information.

Resource Record “A”

The most common RR. It contains the IPv4 address associated with a name.

For example:

alice.example.com	30	IN	A	10.0.0.22

The A RR for the name alice.example.com with a TTL for caching of 30 seconds.

Resource Record “SOA”

The Source of Authority record is always the first record of a zone and it contains administrative information. Each zone must have a different SOA.

Each SOA contains the following RDATA:

• Origin: name of the zone’s primary server.
• Person: e-mail of the zone’s administrator.
• Serial: Integer (YYYY/MM/DD/XX) that must be increased after any modification of the zone data.
• Refresh: time between zone transfer requests by secondary servers (usually days).
• Retry: time between requests whenever a zone transfer fails (usually hours).
• Expire: time a secondary server keeps the data if the connection to the master fails (usually months).
• Negative cache: the time that an inexistent translation may be cached.

If we execute the following command dig @IP to know the RR of our router:

$ dig 192.168.1.1 
 
; <<>> DiG 9.10.6 <<>> 192.168.1.1 
;; global options: +cmd 
;; Got answer: 
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58217 
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 
 
;; OPT PSEUDOSECTION: 
; EDNS: version: 0, flags:; udp: 512 
;; QUESTION SECTION: 
;192.168.1.1.			IN	A 
 
;; AUTHORITY SECTION: 
.			3483	IN	SOA	a.root-servers.net. nstld.verisign-grs.com. 2021041600 1800 900 604800 86400 
 
;; Query time: 147 msec 
;; SERVER: 192.168.1.1#53(192.168.1.1) 
;; WHEN: Fri Apr 16 16:52:20 CEST 2021 
;; MSG SIZE  rcvd: 115

We can see either a type A RR and a SOA, with the following information:

serial: 2021041600

refresh: 1800

retry: 900

expire: 604800

negative cache: 86400

It says that the primary server of the zone is a.root-servers.net and the email of the admin of the zone is nstld.verisign@grs.com.

Resource Record “NS”

This RR stands for Name Server. NS RRs are used for delegation, they link a domain name with the name of an authoritative name server for that domain.

The configuration file of the zone will have as many NS records as domains being delegated in that specific zone.

It tells us which are the servers that serve the other zones that are delegated from the actual zone.

Glue Records

When we are in a certain zone, and we want to delegate to another one, we must indicate which is the name server of that zone with the NS record, as we’ve just seen. The problem is that the NS record does not contain any IP address, and if we are in root, and we want to reach nsc.com, and supposedly the one who knows the IP address of all devices from the zone .com is nsc.com, we will get in a loop.

Thats why we include another “A” record here, called glue record, to indicate the IP address of the server who serves that zone.

Notice that glue records are only necessary if the child zone has a name within the delegated domain, such as .com and nsc.com, then we would need to add a glue record to the .com zone configuration file.

Otherwise, we would only need a NS record (no extra “A” record) if the delegated zone is not related to the parent zone, because it will just send it to root, and it will later delegate it to the correct zone.

Resource Record “CNAME”

When there are several names that translate to the same IP address, we use the CNAME Resource Record. It creates an alias to a canonical name (CNAME).

This RR associates a name with another name of the DNS tree.

Resource Record “MX”

The MX (Mail eXchanger) records designate the mail servers for a given domain. We can add more than one MX record to each mail server for a certain email destination, just keep in mind that they will be contacted by priority order (low number first).

Configuration of DNS servers with BIND

In Linux, we use an implementation of the DNS called BIND (Berkeley Internet Name Domain (BIND), which is maintained by http://www.isc.org.

We will use bind to implement the configuration of our example DNS tree, which emulates a “mini” Internet. In Linux, you can start, stop or restart bind using the command $ /etc/init.d/bind9 stop/start/restart.

Each time you change the configuration of bind, you have to restart it so that changes take effect.

It is also worth knowing that BIND writes its logs by default in a Debian distribution in the file /var/log/daemon.log.

ROOT Servers

To send properly messages between different parts of the DNS tree, we must be able to go “up” the tree, and move to other zones that are not directly connected to the current zone.

That’s why any DNS server must have information about how to reach the root servers of the DNS tree. This informations is called root hints and, in our configuration, they are specified in the file /etc/bind/db.root.

We have configured it this way:

.				IN	NS	ROOT-SERVER 
ROOT-SERVER		IN	A	10.0.0.1

Configuration of ROOT

The root server contains the configuration of the dot zone, which can generally be found at /etc/bind.

When the bind service wants to consult something, the first file it looks at is /etc/bind/named.conf. In root, the contents of this file are the following:

$ cat /etc/bind/named.conf 
 
options { 
directory "/var/cache/bind"; };

zone "." { 
type master ; 
file "/etc/bind/db.root"; 
};

zone "localhost" { 
type master ; 
file "/etc/bind/db.local"; 
};

zone "0.0.10.in−addr.arpa" { type master ; 
file "/etc/bind/db.10.0.0"; };

We can see that for the “.” zone it tells us that the information to reach the “.” zone is inside /etc/bind/db.root

If we take a look at that file, we can see the following:

$ cat /etc/bind/db.root 
 
TTL 60000 ; 16h40m default Time to Live of the 
DNS records . IN SOA ROOT−SERVER. admin−mail .ROOT−SERVER.( 
2006031201 ; serial 
28800 ; refresh 
14400 ; retry 
3600000 ; expire 
0 ; negative cache ttl  
) 
 
.			IN	NS	ROOT-SERVER 
ROOT-SERVER.	        IN	A	10.0.0.1 
 
com. 			IN	NS	nsc.com 
nsc.com.		IN	A	10.0.0.11 
 
net.			IN	NS	nsn.net 
nsn.net.		IN	A	10.0.0.111

Notice that all names end with a dot “.”, that means that all those names are FQDN.

It also tells us which are the name servers that it delegates, and how to reach their zone.

Regarding our example, we can see how this file shows us the different NS and glue records for the three possible delegated zones (“.”, .com and .net).

Configuration of nsc.com

To take a look at the configuration of the server nsc.com, remember that we must look at the file /etc/bind/named.conf of that server.

We have the following in this file:

$ cat /etc/bind/named.conf 
 
options { 
directory "/ var / cache / bind "; 
min−roots 1; 
};

zone "." { 
type hint ; 
file "/etc/bind/db.root"; }; 
.... 
// add entries for other zones below here

zone "com" { 
type master ; 
file "/etc/bind/db.com"; 
};

See how it indicates that we’ve got a hint to reach root at “/etc/bind/db.root”? Exactly how it is supposed to be!

Also notice how it says: “// add entries for other zones below here”, where we can add other child zones that can be connect to this server.

Configuration a zone

Let’s imagine now that we want to observe the configuration of one of the zones connected to the server nsc.com. The .com zone in this case.

To do so, we need to look inside the file /etc/bind/db.”ZONE”, in our case it would be /etc/bind/db.com.

This is what it says:

$ cat /etc/bind/db.com 
 
TTL 60000 ; 16h40m default Time to Live of the DNS records 
com. IN SOA nsc .com. admin−mail . nsc .com. ( 
2006031201 ; serial 
28800 ; refresh 
14400 ; retry 
3600000 ; expire 
0 ; negative cache ttl  
) 
com. 			        IN	NS			nsc.com ; ns of .com

nsc.com.			IN	A		10.0.0.11 ; leaf of .com

bob.com.		30	IN	A		10.0.0.12 ; leaf of .com

example.com.		      IN	NS		nsce.example.com ; delegation of example.com

nsce.example.com	      IN	A		10.0.0.21 ; glue record of example.com

The file contains the TTL, and then the various RRs:

• SOA record with all it’s source of authority information
• NS record of .com zone, indicating who is the name server of .com
• Glue Record of nsc.com telling where nsc.com is
• A record for bob.com
• NS record of the example.com zone, indicating that nsce.example.com is the name server of example.com
• Glue record of nsce.example.com telling where nsce.example.com is found.

If we now take a look at the same file, but from the example.com zone, in the file /etc/bind/db.com.example:

$cat /etc/bind/db.com.example  
 
$ORIGIN example.com. 
$TTL 60000 
@ IN SOA nsce admin−mail . nsce (  
2006031201 ; serial 
28 ; refresh 
14 ; retry 
3600000 ; expire 
20 ; 20 secs of negative cache ttl  
)

@				IN	NS	nsce;	unqualified name

nsce				IN	A	10.0.0.21

david				IN	CNAME	david.example.net.

@				IN	MX	10	mailserver1 
@				IN	MX	20 mailserver2.example.com.

alice				IN	A	10.0.0.22

mailserver1			IN	A	10.0.0.25

mailserver2			IN	A	10.0.0.26

;alice.example.com	        IN	A	10.0.0.22

Let’s take a closer look at this configuration file. The first thing that is different from the previous example is the $ORIGIN line. This is just to indicate that everywhere that a @ appears, it should be substituted by the ORIGIN value, in this case example.com.

Next on we find some RRs that we haven’t seen before, such as CNAME, which creates an alias from another point of the DNS tree, and links this two names to the same IP address, and also MX, which indicates who are the mail servers for example.com.

Finally take a look at the last line… It is commented, but if we uncomment (delete the semicolon) from that line, we are creating another A resource, and it renames the zone that is found at the IP address 10.0.0.22 to alice.example.com instead of alice.

DNS Queries & Responses

When a Linux device tries to find a translation for a name, the first thing it does is looking at the local translations file located at /etc/hosts.

If there is not any translation in this file, the client has to use the DNS service.

To do so, we need to know at least one name server. In Linux, the file /etc/resolv.conf is where the addresses of the name servers are stored.

The contents of the file are something like this:

nameserver 147.83.2.3 
nameserver 147.83.2.10

They are pointing to different name servers, to which the client will ask for DNS translations. In the resolv.conf you can set up to 3 name servers that will be contacted in order.

Obviously, the name servers to which you ask, must be configured to accept client queries.

Additional note: The name server is typically provided by your ISP or your organization.

Queries

In Linux we have 2 commands to perform DNS queries: host and dig.

Each query specifies the domain name, class and type of the queried RR. For example we can ask for klasea.es IN A.

There are 2 types of queries: iterative and recursive.

• In an iterative query, the queried name server returns the address of the next name server that you must consult to obtain the response.
• In a recursive query, the name server tries to find the final response, making several queries to different name servers if necessary. To use the recursive queries, we must activate the flag “recursion flag” on the request.

Be careful! Not all servers support recursion.

Responses

Name servers usually use a cache memory to improve the DNS performance. The time that a response is cached is configured by the original Source Administrator.

When we use cache, we have to possible types of responses:

• Authoritative: Authoritative name servers provide authoritative responses (not cached) because it is the original source.
• Non-authoritative: Non-authoritative name servers provide cached responses (with a valid TTL).

DNS Protocol

DNS servers use by default UDP port 53 , and DNS messages can have five parts:

• Header: here we can find if the message is a query or a reply, if recursion is desired or not, if the response is authoritative or not, etc.
• Questions: They are tuples with Name, Type and Class.
• Answers: Contain the registers that match the Name, Type and Class asked in the questions part.
• Authority: It typically contains NS records pointing to name servers closer to the target name in the hierarchy.
• Additional information: It contains additional records that the name server believes may be useful to the client. The most common use for this field is to supply A address records for the name servers listed in the Authority section.

Practical Name Resolutions

Querying an Authoritative Server

Normally clients will be asking queries to authoritative servers.

In our example let’s consider that Alice has nsce as her resolver (can be found in /etc/resolv.conf) and that she wants to know its own IP address.

To do so Alice will send a recursive query for the A register of alice.example.com to nsce.

Then nsce will send directly the response to Alice, as it already knows the answer.

The command Alice had to use to know her own IP address is: alice$ dig alice.example.com, and the response she obtains is the following:

; <<>> DiG 9.6-ESV-R4 <<>> alice.example.com 
;; global options: +cmd 
;; Got answer: 
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49052 
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: 
;alice.example.com. IN A 
;; ANSWER SECTION: 
alice.example.com. 60000 IN A 
;; AUTHORITY SECTION: 
example.com. 60000 IN NS 
;; ADDITIONAL SECTION: 
nsce.example.com. 60000 IN A 
;; Query time: 289 msec 
;; SERVER: 10.0.0.21#53(10.0.0.21) 
;; WHEN: Tue Mar 26 11:29:34 2013 
;; MSG SIZE rcvd: 86

Let’s examine the output, if we take a closer look at the flags it has we can see:

• aa: authoritative response
• rd: recursion desired
• ra: response available

If we now made a dig to a non-existent target:

alice$ dig alan.example.com 
; <<>> DiG 9.6-ESV-R4 <<>> alan.example.com 
;; global options: +cmd 
;; Got answer: 
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 39136 
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: 
;alan.example.com. IN A 
;; AUTHORITY SECTION: 
example.com. 20 IN SOA nsce.example.com. ...

As you can see the server returns a response with status NXDOMAIN, which means that a translation for that name was not found. It also returns a SOA RR, because it contains the negative TTL, which tells us how long we must store this information in the cache. In this case 20 seconds.

Non authoritative server

Until now we’ve seen how a server reacts when it is the authoritative server of the client who asked the query.

But what happens when a client wants to know the information of a host, who is not in the same zone, and so the previous server is not the authoritative server of the new zone?

Taking our example back, let’s imagine that from Alice, we want to find information about bob.com. The authoritative server of bob is nsc, in this case nsce does not know where it is.

We will test what happens using the dig command from alice, and capturing the traffic with wireshark (we will use the option Statistics -> Flow Graph).

$alice dig bob.com

Let’s analyze what happens here.

• (1) First alice sends the query to its server, nsce, asking only for the A record.
• (2) nsce doesn’t know where to find bob.com, so it sends the query to it’s root server. In this case it asks for either the A record and the NS.
• (3) root responds with the NS and A records from either nsc and its own.
• (4) Now nsce knows that the NS of bob.com is nsc, and so forth it sends the query to nsc.com asking for the A record from bob.com.
• (5) nsc responds to nsce sending the A record of bob.com
• (6) nsce can tell now to alice where bob.com is located.

Take a look at the following image for a visual representation of the previous steps:

DNS caching

Name servers use caching to enhance the performance of the DNS system by storing the RRs obtained during the resolution process. These RRs are kept in cache until their TTL expires. A non authoritative server can use its cached RRs with valid TTL to answer its clients queries without contacting any other name servers.

Imagine that alice (or another user) asks again nsce for the A record of bob.com. As it still has it in its cache, then nsce can directly respond the query with its cached record.

Take into account that different RRs have different TTL, so maybe the A resource of bob.com is already expired, but the NS record that tells nsce that nsc is the name server of bob.com has not, and so nsce could ask directly to nsc for the A record of bob, without need to go to the root server.

Non Recursive Questions

If we try an iterative query with the dig command from alice to nsce, about bob.com, the input will be the following:

alice$ dig +noquestion +norecurse bob.com 
; <<>> DiG 9.6-ESV-R4 <<>> +noquestion +norecurse bob.com 
;; global options: +cmd 
;; Got answer: 
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30988 
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; AUTHORITY SECTION: 
. 0 IN NS ROOT-SERVER.

As you can see, nsce returns the NS record of the ROOT-SERVER, which is the next server you have to contact to resolve bob.com.

If alice now asks root for bob.com:

alice$ dig +noquestion +norecurse @10.0.0.1 bob.com 
; <<>> DiG 9.6-ESV-R4 <<>> +noquestion +norecurse @10.0.0.1 bob.com ; (1 server found) 
;; global options: +cmd 
;; Got answer: 
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23767 
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; AUTHORITY SECTION: 
com. 60000 IN NS nsc.com. 
;; ADDITIONAL SECTION: 
nsc.com. 60000 IN A 10.0.0.11

We obtain the NS and A records of nsc which is the next server to resolve bob.com.

Great! Up to this point, we’ve been explaining all the basics to know how the DNS service resolves queries, and goes from one server to another.

The following sections are just extra information that imo are worth to mention, but in case you already found what you were looking for, or you just wanted to understand the DNS protocol, maybe you won’t need them, though I recommend you to take a look if you already reached this far!

Reverse Lookups

As you know, DNS allows to translate a FQDN to an IP address.

Reverse lookups allow exactly the opposite, translating IP addresses into FQDNs.

Some applications use the reverse translations as a measure of security, but the majority do not use them, as it is not compulsory.

The RR used for reverse lookups is called PTR (pointer), and as you may have guessed, it points to the corresponding FQDN.

If you want to ask for a reverse translation, you can use the dig command with the -x option: dig -x 147.83.2.135, and it will answer all the FQDNs that correspond to this IP address.

Other RRs

In this article I’ve explained the most common and compulsory RRs that the DNS uses. However, there are other ones that you may find

I’ve not went in depth with them because they are not as common as the ones we’ve seen, but just keep in mind that you may find additional RRs like:

• TXT: plaintext record containing optional information.
• HINFO: contains informative plain text about the server hardware and OS.
• LOC: contains geographical location of the server.
• AAAA: RR for IPv6 indicating the whole v6 address.
• SRV: allows specifying the location of the servers of a certain service.

Real Root Servers & Administration Authorities

In our example we’ve been using a “mini Internet” with only one root server. Obviously the real Internet does not have only one root server.

Initially there were 13 root-servers located in different countries.

Nowadays we still have 13 names for root servers which are specified in the form letter.root-servers.net, where letter ranges from A to M.

However this does not mean that we have only 13 physical servers. For reliability and performance, each name server is implemented using redundant computer equipment to be able to provide DNS service even if failure of hardware or software happens in one of the physical servers.

Finally, after talking about the different servers that we have around the world, it is worth mentioning the Internet Corporation for Assigned Names and Numbers (ICANN³), who is the responsible of managing different network parameters such as:

• Root and first level name servers
• Assignment of IP addresses
• Delegation of zones to RIRs (Regional Internet Registries)

Well, I guess this is a good starting point to understand the overall picture on how does DNS work. I hope you understood everything, and found it useful.

If you want to take a look at a practical exercise step by step on how to use DNS, using the same pictures and situations of this article, you can do it here!

For any doubts, hit me up at akakush19@protonmail.com, I’ll be happy to share thoughts with you!

If you want to check my previous articles to understand some other Internet concepts feel free to do so!

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics
• Chapter 3: Network Apps

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

Welcome to the third chapter of the Internet Educational Series.

Here you’ll learn all the basics from the Internet, how does it work, why does it work, the protocols used, and many more you’ll discover as you read through my articles.

I hope you like them, and enjoy them as much as I’m doing when writing them.

Let’s get into it! Today we are talking about the Basic Network Applications, how are they built from scratch, and some examples on how to use them.

I’ll start from the beginning, trying to make everything understandable for everyone, even if you haven’t got a tech background.

If you want to check my previous articles to understand some other Internet concepts feel free to do so!

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics

Transport Layers

Transport Layer Motivation

The network layer is a scalable way of interconnecting data link layer technologies.

Basic IP provides an interface (NIC-to-NIC) best effort service for delivering datagrams, which means that a correct delivery of datagrams is not guaranteed, there might be lost datagrams, incorrect datagrams or disordered datagrams.

The main goal was to implement communications between processes that run generally in different systems, commonly known as end-to-end communications.

Here the concept of PORT was introduced, for multiplexing and demultiplexing this communications.

The port concept led to working in client-server models for each once a communication was established. This means that there is a server who is running a daemon in the background, executing the process continuously, and users, who act as clients, make requests to this servers to access the information they want.

To identify the different processes that are running, Unix-like systems use the Process Identifier (PID), however for multiplexing transport communications we want a generic identifier, which is the actual PORT.

So, a port is just a parameter for multiplexing that is dynamically assigned to any running process that requires a transport communication with another process.

Each transport PDU carries:

• A source Port (SPort) that identifies the process sending the PDU
• A destination Port (DPort) that identifies the process in the destination host

We are now going to see the main transport protocols used on the Internet.

User Datagram Protocol (UDP)

• UDP is the simplest transport protocol
• It is a message-oriented protocol (datagram protocol)
• Each UDP message is encapsulated over an IP datagram
• UDP only offers multiplexing capabilities (using ports) and a checksum for discarding wrong data
• UDP does NOT provide error, flow or congestion control

Transport Control Protocol (TCP)

• TCP provides applications with a full-duplex communication, encapsulating its data over IP datagrams
• TCP is connection-oriented because there is a handshake of three messages before data can be sent
• Apart from multiplexing capabilities, TCP is a reliable protocol because it adds support to detect errors or lost data and retransmit them (ARQ end-to-end error control)
• TCP also supports end-to-end flow control and congestion control

With this two protocols, we can make sure that our data is properly sent, and check that the channels where we want to send the data are accessible before sending it.

Client/Server model

As we have seen previously, the client/server model is the most widely used model for communication between processes. Clients make requests to the servers, and servers respond and they can generally support numerous clients.

In Unix-like systems, servers processes are also called daemons. In general a daemon is a process that runs in the background, and typically they have names that end with the letter “d” (e.g: telnetd, ftpd, httpd…)

Clients initiate the interprocess communication, so they must know the address of the server.

Luckily, there are a list of well-known ports.

The client usually knows the IP of the server, and there is a well-known transport protocol and port determined by the application used.

For example, HTTP servers (for the Web) use TCP/80, DNS servers use UDP/53 for name queries, and the DHCP servers use UDP/67.

The command netstat (network statistics) shows established or listening sockets and several related statistics.

Socket API

TCP/IP communications were developed in the context of Unix systems. One of the main ways of implementing TCP/IP communications is to use the “socket” API.

An application programming interface (API) is an interface implemented by a software program to enable interaction with other software.

It may include specifications for routines, data structures, object classes and protocols used to communicate between the consumer and implementer of the API.

Netcat

The netcat application can be used to create a process that opens a raw TCP or UDP socket as a client or server.

It is so useful for testing networks, that it even is known as the “Swiss Army Knife of networking”.

• If we want to use netcat as a client: nc hostname port
• If we want to use it as a server: nc -l -p port —> we add the -l parameter, which stands for “listening”.

Note that a netcat server is NOT multiclient.

We can use netcat too to transfer files, create remote terminals, take a look at open files, etc.

Let’s see an example:

If we want to transfer files, we can create a netcat server in the device we have the files stored, and then run a client connection in the device we want to transfer the data to:

server# cat file.txt | nc -l -p 12345 -q 0

The -q 0 option closes the connection once the file has been transferred.

Now we can create a client connection by tiping:

client# nc hostname 12345

Basic Network Applications

Great, now you know how the transport layers work and how we can establish connections between clients and servers, so let’s dive into some of the basic applications that use this technologies.

In this section we will be talking about:

• TELNET
• FTP
• SSH
• Super Servers

TELNET

What is TELecommunication NETwork?

TELNET is a standard Internet protocol for emulating a terminal in a remote host using an IP network.

The TELNET service uses TCP, and it is a well-known port, that uses port 23.

To use the client, you just need to open a terminal and type telnet. After you type the command, you will enter the telnet “sub-shell”, where you’ll be able to type specific commands of telnet.

The most used and useful subcommand is open hostname, which starts a connection with the specified IP address or hostname with the port 23 (this is the default port, but you can specify a different port after the IP address).

You can also type the subcommand help to obtain a list of possible subcommands.

Let’s see a practical example on how to open a TELNET connection.

$ telnet 
telnet > open 192.168.0.1

Note that you need to be in the same network as your server to be able to access it.

You can also use the parameters directly in the command line:

$ telnet 192.168.0.1 23

You can exit the remote terminal when you want by typing “exit”.

Things to keep in mind with TELNET:

Many times telnet clients are used to establish an interactive raw TCP connection. Despite most of the times there will be no problems with this, it mush be taken into account that the telnet client can alter the data sent, as the value 255 is altered. If you want a pure raw TCP connection, you should use the netcat tool.

Finally, it is worth to mention that because of security issues with telnet, its use for the purpose of having a remote terminal has waned in favor of SSH.

FTP

What is a FTP (File Transfer Protocol)?

FTP is a standard Internet protocol for transmitting files between computers (hosts) on the Internet. FTP uses IP and it is based on a client-server model.

FTP uses separate control and data connections between the client and server.

The default server port for control data is 21, and the default server port for data exchanging is 20.

The control connection remains open for the whole duration of the session, and it is used for session management (commands, parameter negotiations, passwords, etc.) exchanged between the client and the server.

Due to this 2-port structure, FTP is considered an out-of-band, as opposed to in-band protocols such as TELNET.

The server responds to control connection with three digit status codes in ASCII, with an optional text message, for example “200 OK” means that the last command was successful.

Active and Passive modes

The mode FTP is run, determines how te data connection is established.

• In active mode the client sends the server the IP address and port number on which the client will listen, and the server initiates the data connection.
• In passive mode the client sends a PSAV command to the server, and receives an IP address and port number in return.

Data transfer modes

Data transfer can be done in any of three modes:

• Stream mode: Data is sent to a continuous stream, relieving FTP from doing any processing. All processing is left to TCP.
• Block mode: FTP breaks the data into several blocks (block header, byte count and data field) and then passes it to TCP.
• Compressed mode: Data is compressed using an algorithm

Practical FTP

Next we discuss the ftp command line client and ftpd server for Linux.

The ftp client supports active and passive mode, and only stream mode.

To connect to a ftpd server we can use one of the following options:

$ ftp name 
$ ftp 192.168.0.1 
$ ftp user@192.168.0.1

To establish an FTP session you must know the ftp username and password. In case the session is anonymous, typically, you can use the word “anonymous” as both username and password.

When you enter your own login name and password, it returns the prompt ftp>, which is a subshell where you can type several subcommands.

Finally, it is worth to mention that you can also use FTP through a Browser, like:

ftp://ftp.upc.edu 
ftp://ftpusername@ftp.upc.edu 
ftp://ftpusername:password@ftp.upc.edu

SSH

Secure SHell is a protocol used to obtain an encrypted end-to-end TCP connection between a client (ssh) and a server (sshd) over a TCP/IP network. The sshd daemon listens to port 22 TCP by default. SSH encrypts all traffic, which ultimately eliminates eavesdropping, connection hijacking, and other attacks.

Services with SSH

Remote Terminal

The first example is to use ssh client to connect to a sshd server to obtain a remote terminal. The same idea as TELNET, but secure.

You can achieve this by typing: user1$ ssh 192.168.0.1

The previous command will try to establish a SSH session over a TCP socket between the client ssh and sshd server listening on 192.168.0.1:22.

By default, SSH assumes that you want to authenticate with the user you are currently using.

If you want to use a different user for login on the remote host, simply use: remoteusername@hostname like this:

user1$ ssh user2@192.168.0.1

With SSH we can also transfer files securely between machines on a network.

We can do this with SCP and SFTP.

SCP (Secure Copy)

The command scp is essentially a client program that uses the SSH protocol to send and receive files over and encrypted SSH session. You can transfer files from the client to the server and vice versa.

Let’s see a simple command that copies a file called file.txt from the client to the server, to the home directory of the user “username”:

$ scp file.txt username@remotehost:

If you want to specify a new directory and a new name for the file, you can do it this way:

$ scp file.txt username@remotehost:mydirectory/anothername.txt

To recursively copy a directory to the server, use the “-r” option:

$ scp -r Documents username@remotehost:

And finally, to copy from the server to the client just reverse the from and to:

$ scp username@remotehost:file.txt

SFTP (Secure FTP)

SFTP is a secure implementation of the traditional FTP protocol using a SSH session. Let us take a look at how to use the sftp command:

$ sftp user@hostname

Start/Stop sshd

Each time you change the configuration of sshd you have to stop and start it to apply the changes.

You can do it this way in most Linux systems:

# /etc/init.d/ssh stop 
# /etc/init.d/ssh start

Super Servers

A way of implementing a network service is to use a stand-alone daemon. A stand-alone daemon listens to a port and serves client requests associated with that port.

However, for simple services that are used not very often it is more efficient to use a “super server” or “super daemon”.

A super-daemon listens to several ports. They use memory more efficiently.

On the other hand, a dedicated or stand-alone server that intercepts the traffic directly is preferable for services that have frequent traffic.

In Linux inetd is the typical super-daemon.

Each time a client requests a service managed by inetd, the super-daemon uses its configuration file (/etc/inetd.conf) to determine which server program should manage the request and executes the corresponding server program.

• For each TCP connection, inetd creates a process and connects STDIN and STDOUT of this process to the socket established with the client.
• UDP sockets are generally handled by a single application-specific server instance that handles all packets on that port.
• Some simple services such as “daytime” are handled directly by inetd, without spawning an external application-specific server.

Configuration

The inetd super-daemon is configured using the file /etc/inetd.conf .

Each line in the file contains the following fields:

service_name	socket_type	{wait|nowait}	user	server_program	[server_program arguments]

For example, the configuration line for the TELNET service is something like:

telnet	stream	tcp	nowait	root	/usr/sbin/in.telnetd

Otherway, service names and ports are mapped in the configuration file /etc/services.

You can check the default port for the telnet service typing:

$ cat /etc/services | grep telnet 
 
telnet 		23/tcp 
...

The last file important to mention is the /var/log/daemon.log, where inetd writes its messages.

Start/Stop inetd

Each time you change the configuration of inetd you have to stop and start it to apply the changes. It can be done as follows:

# /etc/init.d/openbsd-inetd stop 
# /etc/init.d/openbsd-inetd start

Creating a Service under inetd

Follow this steps to create a service under inetd:

• Create the server program using STDIN and STDOUT.
• Choose a name and a port for the service.
• Set the port/name of the service in the /etc/services file.
• Start and stop inetd

Let’s configure the service “my-echo” following the previous steps:

1. Add the configuration file in /etc/inetd.conf
2. my-echo stream tcp nowait root /root/my-echo.sh
3. Restart inetd and observe the log file where inetd writes its messages:
4. # tail -f /var/log/daemon.log
5. inetd[1173]: my-echo/tcp: unknown service
6. Add a port number for our service in the file /etc/services:
7. my-echo 12345/tcp
8. Restart inetd and try to connect to the service. Let’s observe the log file
9. # tail -f /var/log/daemon.log
10. execv /root/my-echo.sh: No such file or directory
11. We need to create the file as it indicates in the previous message:
12. cat

The command cat without parameters, writes to STDOUT what it reads from STDIN.Let’s take a look again at the log file:

# tail -f /var/log/daemon.log 
inetd[1173]: my-echo/tcp: Permission denied

1. Give permisions to the script, and get back to the logs file:
2. chmod u+x /root/my-echo.sh
3. #tail -f /var/log/daemon.log
4. inetd[1207]: execv /root/my-echo.sh: Exec format error
5. We need a “formally correct” script, and to do so we need to add the following initial line with the corresponding interpreter
6. #!/bin/bash
7. cat

We will now create a netcat client connection to check that the my-echo service is working properly.

You can observe the connections of the services with the lsof command, and check that the STDIN, STDOUT and STDERR of the processes are connected to the corresponding socket

virt1# lsof -a -p 1216

and we see that the 0, 1 and 2 FD are established correctly.

Voilà! We’ve got the service configured.

Alright, I know this has been a long journey, but congrats if you made it up to this point! Take your time to digest everything we’ve been seeing so far, and if you still have any doubts, hit me up at akakush19@protonmail.com, I’ll be happy to share thoughts with you!

If you want to check my previous articles to understand some other Internet concepts feel free to do so!

• Chapter 1: Ethernet, Switching & VLANs
• Chapter 2: IP Basics

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

Hello dear reader! And welcome to my Internet Educational Series.

Here you’ll learn all the basics from the Internet, how does it work, why does it work, the protocols used, and many more you’ll discover as you read through my articles.

I hope you like them, and enjoy them as much as I’m doing when writing them.

Let’s get into it! Today we are talking about the Internet Protocol Basics, also know as IP.

I’ll start from the beginning, trying to make everything understandable for everyone, even if you haven’t got a tech background.

1. Motivation. Why a Network Layer?

When the first Ethernet networks were created, people started thinking on how we could expand, and connect more and more people to those networks.

That’s when the two main problems, that the Internet tries to solve, appeared:

1. Universal Interconnection
2. Scalability (Up to millions of interconnected devices)

And today you’ll learn how humans have solved this two problems, with the different technologies involved, and how they interact between them to provide the global interconnected network that we know today as The Internet.

The first step to get universal interconnection is to have a global network of addresses to identify devices connected to the different data link layers.

Many different network protocols have been implemented: IP, IPX (from Novell), NetBIOS (from Microsoft), etc.

Fortunately IP (Internet Protocol) has become almost the unique network protocol used.

IP has 2 versions:

• IPv4 (the one we will talk about today)
• Network addresses of 32 bits
• Addresses are represented with the dot-decimal notation: 192.168.1.0
• Each bytes is represented in decimal with a value between 0 and 255.
• IPv6
• Network addresses of 128 bits.

To send data using IP to a certain device, you need to know its IP address, but IP addresses are hard to remember.

That’s why we use DNS (Domain Name System) to translate addresses to names. But we will talk about this in another article.

Devices also have physical addresses, which are the MAC ones, as we saw in the previous article about Ethernet. Check it out here if you have not read it.

This MAC addresses are hard-coded by the vendor to identify the device uniquely.

However, only with the MAC addresses we cannot scale to big networks, and that’s why Ethernet cannot be deployed as a universal network, and we need something else.

2. IP Addressing

IP implements a “clever way of assigning addresses”.

First of all, it is important to know that IP addresses are configured, not hard-coded.

You can use the ifconfig command to configure an @IP to a NIC (Network Interface Card).

Example of a configuration of an IPv4 address in Unix Systems:

ifconfig eth0 192.168.0.1

The most important features of IP are:

• Devices that are “close” must have “similar addresses”, which mean addresses with a common prefix.
• IP traffic is routed using prefixes, instead of single addresses, which allows reducing the size of the routing tables.

It is very usual to get our IP address configured dynamically from a local server with a protocol called DHCP (Dynamic Host Configuration Protocol), though we’ll talk about it in later articles.

2.1 NetID & HostID

IP Divides the address in two parts to implement the network addressing plan:

• Devices with a common NetID are said to be in the same IP network.
• Devices in different IP networks must have different NetIDs.
• The portion of the address that identifies a particular device on the network is called the HostID.

For now we will consider that the address division is 24/8: 24 bits for NetID and 8 bits for the HostID.

For example if we have the IP address 192.168.0.1:

• NetID: 192.168.0 (24 bits)
• HostID: 1 (8 bits)

Note: There are some addresses that are reserved in each Network that depend on the HostID.

• If the HostID = 0s it is reserved for the network address.
• If the HostID = 1s it is reserved for the broadcast address.

So in our example, 192.168.0.0 would be the network address, and 192.168.0.255 would be the broadcast address.

But how do we separate this addresses between them, you may ask?

The answer is with the devices we all need at home if we want to connect to the Internet. Routers are the ones who separate the different IP networks.

And this is how we get to understand the next step.

3. Routing

Routing is the name we have given at the process that routers do when interconnecting different data link layers.

Routers transfer IP packets from one DLL to another, but frames are not transferred.

When doing IP Routing, what we are doing is searching all over the Internet the destination device we want to reach.

To do that, IP users provide the destination IP address or name to the router, and this one tries to find where it is located, to transmit data between the two hosts.

However, IP does not guarantee a reliable delivery. It provides a best effort host to host service, as it uses a hop by hop routing mechanism.

This means that each packet is sent from one router (hop) to another one (next hop) in the path.

By using this mechanism, we achieve a higher scalability than with Ethernet for various reasons:

• Routing tables group addresses: The amount of information that a router needs to keep is proportional to the number of networks in the Internet, not the number of devices.
• Routers filter Layer 2 traffic: they efficiently forward (route) packets between data link networks.

Example of IP Routing:

4. Internet Applications

Many protocols over the Internet use the IP network with a client/server model.

But what is a client/server model exactly?

• Servers are processes or network daemons that are running continuously in the background.
• Clients are the ones who initiate the communication with servers.
• Clients must know the address of the server (but not necessarily otherwise when the communication is initiated).

When we initiate processes, the address must be composed of 3 parameters:

IP address, transport protocol and port.

The transport protocols (L4) provide multiplexing identifiers for their users. This multiplexing IDs have a size of 16 bits and they are called ports.

There are two transport protocols: TCP and UDP.

• User Datagram Protocol (UDP)
• Simplest transport protocol
• Message-oriented
• Each UDP datagram is encapsulated in a IP datagram
• UDP only offers multiplexing and checksum for discarding wrong data
• For multiplexing it uses the source and destination ports
• Transport Control Protocol (TCP)
• TCP provides full-duplex communication, and encapsulates its data over IP datagrams
• Connection-oriented (there is a handshake of three messages before data can be sent)
• The communication is managed as a data flow (not message-oriented)
• TCP is reliable, as it adds support to detect errors or lost data and to trigger retransmission until the data is correctly and completely received

Over this protocols, we do have the next Layer in the OSI model, which is the Application Layer.

Here we can use the different applications that exist over the Transport protocol layer, like WHOIS, HTTP, DNS, DHCP…

A nice recap of what we’ve seen so far would be the following image:

5. Classful IP

Let’s get back to the backbone of this article.

We’ve seen before that IP addresses are divided in NetID and HostID, to identify either the network the device is in, and the exact device that we want to establish a communication with.

Let’s now see how does IP solve the problem of finding who is each device in their network.

5.1 Address Resolution Protocol (ARP)

When we want to find who is exactly each device in our network, we use the ARP protocol.

ARP is a request/response protocol for the dynamic resolution of the mapping @IP-@MAC.

It uses two messages ARP-request and ARP-reply, both encapsulated in MAC frames.

To figure out the MAC address associated with a certain IP address, ARP does the following process:

• The sender sends an ARP-request to broadcast (ff:ff:ff:ff:ff:ff)
• This request contains the IP address of the host whose MAC address we want to know.
• The device in the L2 network that has been configured with the IP requested, sends an ARP-reply.
• This ARP-reply is unicast and it has as destination MAC the @MAC of the station that sent the ARP-request.
• Now, both sender and receiver know the L2 and L3 address mappings of each other.

ARP Caché:

Since Broadcasting is very costly, a local file is used to store the bindings learned with ARP.

With this caché, we can handle future requests faster or more efficiently.

However, we need a mechanism to update this cache, because it cannot store all the addresses forever, as some addresses turn obsolete.

This mechanism is a timeout, or Time To Live (TTL). When the TTL expires, the binding is removed from the ARP cache.

TTL can be manually configured.

In a Lunix OS, we can use several commands to view the ARP cache and to re-configure it, for example:

arp -n 						#View the arp cache 
arp del 192.168.1.2 		#Delete mapping 
arping 192.168.1.2			#ARP request 
ifconfig 					#View interfaces configuration 
ifconfig eth0 192.168.1.1	#Configure IP to eth0 
arp set 192.168.1.2 00:23:ae:1c:51:29 [temp] #sets a temporal mapping

So far, we’ve learned how IP packets are exchanged between devices that are on the same data link.

But we want to exchange packets between devices that are not in the same data link.

So, how can we know that a device is connected to a certain data link?

The answer is the 2 parts of the IPv4 addresses, NetID and HostID.

As you know, the devices that share the same data link, must have the same NetID, so forth, a device only needs to check if the destination @IP has the same NetID to know if the receiver is in the same data link of the sender, or on another one.

Now we need to introduce a new concept to understand how to route the packets to the correct data link.

Classes of Addresses.

Class-A Addresses

• 8/24
• NetID: w → 0–127
• HostID: x.y.z
• Number of Networks: 2⁷ = 128 networks
• Number of addresses per network = 2²⁴ = 16.777.216 addresses

Class-B Addresses

• 16/16
• NetID: w.x. → 128–191
• HostID: y.z
• Number of Networks: 2¹⁴ = 16384 Class B networks
• Number of addresses per network = 2¹⁶ = 65536 addresses
• Private addresses from 172.16.0.0 to 172.31.0.0

Class-C Addresses

• 24/8
• NetID: w.x.y → 192–223
• HostID: z
• Number of networks: 2²¹ = 2097152 Class C networks
• Number of addresses per network: 2⁸ = 256 addresses
• Private network addresses from 192.168.0.0 to 192.168.255.0

Class-D and Class-E

Class D:

• Used for group communications, or multicast.
• The first byte of class D goes from 224 to 238.

Class E: the whole addressing of this class is reserved for future use

6. Types of Delivery

Great, so now you know how are addresses routed to get to their destination. To classify in an easier way the routing of datagrams, we introduce two more concepts called

• Direct Delivery
• Indirect Delivery

6.1 Direct Delivery

We use direct delivery when either sender and receiver are physically connected to the same L2 network.

This means the destination IP address has the same NetID than the IP source address.

Direct deliveries do not require intermediaries (routers).

We use the ARP cache or the ARP protocol to find out the destination @MAC, which is the only missing parameter to build the MAC frame.

6.2 Indirect Delivery

The next step is to learn how to make indirect deliveries, which as you may have guessed, are the types of datagram deliveries that are made from one device that is connected to a certain network, to another device that is connected in a different network.

For this purpose, we use routing tables and an intermediate device: an IP router (IP gateway).

A routing table is a local table (each host or router has one) that is consulted to find the parameters of the next hop, where the packets must be forwarded.

In Unix-like systems you can view the routing table with the command route -n.

The first thing we need to observe in the previous example, is that our router has two NICs and also two IP addresses configured, one to each NIC.

In general, routing tables contain at least the following entries:

• Destination network address
• Output NIC
• Next Hop (The @IP of the next hop)

Let’s see an example of the previous picture, but with it’s corresponding routing tables from each host:

So, once you’ve understood the routing tables, and how they are configured into each host, it’s much easier to get how indirect deliveries work.

IP routing is performed hop by hop: routers/hosts only take care of finding the parameters of the next device where to forward the packet.

So for this purpose, the routing table provides us with the @IP of the next hop, which has to be directly reachable.

6.3 ICMP

As you may imagine, indirect deliveries have more problems than direct ones.

That’s why ICMP was implemented. This protocol is called Internet Control Message Protocol (ICMP), and its purpose is testing and finding anomalies in networks.

Let’s imagine that a packet arrives to a router that hasn’t got any routing entry for the destination.

In this situation, ICMP defines a message that the routers can send to the source indicating why the packet did not arrive to its destination and why it has been discarded.

Then we can find another better route to send our packet through, or try to solve the problem.

If you ever try to do networking tasks, you’ll find ICMP very useful to test if the network is working or not, and you’ll get very used to the following messages:

• echo-request (ping)
• echo-reply (pong)

7. Classless IP

The previously explained IP (classful IP) has two big problems:

1. Address Exhaustion
2. The class structure is not efficient
3. An institution or company that wants to build an IP network of 6 hosts, needs an entire C-class address, which will result in 248 unused addresses. That scales even worse when we need more addresses, as if we need to connect 300 hosts, we would have to ask for a B-class, and would end up in 16082 unused addresses.
4. Slow Routing
5. Classful addressing generates large and unmanageable routing tables, as we need one entry in the routing table per class.
6. This was critical for the routers of the Internet Backbone that have to route packets at high speeds.

That’s when people realized that Internet was “dying of success”, and something had to be done.

The proposed solutions for this problems were the following ones:

• Subnetting / Supernetting
• CIDR (Classless Inter Domain Routing) with geographical allocation of addresses.
• Dynamic NAT for using less public addresses.
• DHCP for reusing IP addresses

Moreover, IPv6 is another solution, which we will not cover in this article, but also solves all the previous problems and has an even better scalability.

It’s main difference with IPv4 is the address space, which goes from 32-bit to 128-bit addresses.

7.1 Subnetting

When dealing the previously seen problems, a new concept appeared: Net masks.

Net masks allow us to split networks in many different parts, and also group them in bigger ones.

In particular, a Subnet is a network smaller than the original class.

To indicate that a certain network has a net mask, we can use different notations, however the most common one is the CIDR notation.

This notation is exactly like this: @IP/X where X is a decimal value indication the number of 1s of the network mask.

Eg 1: 147.83.0.0/16 is a network number of a class B network.

Eg 2: 192.168.1.3/24 is an IP address of a host in a class C network.

Let’s see an example of a Class C Division (the most common ones you’ll see)

We want to divide the private Class C 192.168.1.0, which has room for 256 addresses, into 2 different subnets with 128 addresses each.

Actually, subtracting the network address and the broadcast address from each one of them, we would end up with 2 networks of 126 hosts each.

To do so, we need extend the network part in one bit.

Now I’ll show you my way of calculating how to identify the netmask we need to divide our bigger networks.

As we know in Class C addresses, we have 256 free addresses, which is 2⁸.

This means we have 8 bits for the HostID (256 hosts), and 24 bits for the NetID.

If we add one extra bit to the NetID, we end up with 25 bits for the NetID and 7 bits for the HostID.

If we calculate how many hosts we can have with 7 bits, we just do 2⁷ = 128.

Exactly the number of hosts we needed!

So now, we can see our 2 subnets, divided thanks to our mask:

1. The first one goes from 192.168.1.0/25 → 192.168.1.127/25
2. The second one goes from 192.168.1.128/25 → 192.168.1.255/25

Masks are also used in the routing tables of routers that are “aware of subnetting”.

Let’s see an example to understand it better:

Notice that we might have an IP network in which we have classful routers and classless routers (r1 and r2).

Outside the subnets, the routers can still use classful routing.

In our example, router 1 still sees 192.168.1.0 as a class C network address, and uses a classful routing table.

However, the internal router (2), is aware of subnetting and it is able to differentiate between the two subnetworks.

In this case, to properly route IP packets to each subnet, the router 2 uses a classless routing table, which includes masks for correctly interpreting IP network numbers.

Until now, we have been using a type of subnetting called FLSM (Fixed Length Subnet Mask), which as it name indicates, it creates fixed subnets, all of them have the same number of hosts.

But what happens if we want to divide a /24 network (256 addresses) into 3 subnets, one with 120 hosts, and the other 2 with 60 hosts each one?

Here is when VLSM (Variable Length Subnet Mask) comes into play.

To divide networks into smaller ones, we must always start with the biggest one we want to create. So in this case, we would first need to assign 7 bits for the HostID (120 hosts needs at least 128 addresses, 2⁷) and then we would need at least 2 extra bits to differentiate the 3 networks between them.

So we would end up with 9 bits needed for this network, which makes it imposible for a /24 network to divide it when using FLSM. We would need at least a /23 network, and the subnets would be like this:

We can see that we need 9 bits to divide the 3 networks (which will actually create 4 networks, and one of them will be redundant).

• SubnetID == 00 → 10.0.0.0 to 10.0.0.127 (/25)
• SubnetID == 01 → 10.0.0.128 to 10.0.0.255 (/25)
• SubnetID == 10 → 10.0.1.0 to 10.0.1.127 (/25)
• SubnetID == 11 → 10.0.1.128 to 10.0.1.255 (/25)

On the other hand, if we used VLSM, we could have adapted our network with just a /24 mask, as the 8th bit would actually differentiate the bigger network from the smaller ones, and the 7th one would differentiate the two small networks between them.

• SubnetID == 0 → 10.0.0.0 to 10.0.0.127 (/25)
• SubnetID == 10 → 10.0.0.128 to 10.0.0.191 (/26)
• SubnetID == 11 → 10.0.0.192 to 10.0.0.255 (/26)

This way we would have room for all the hosts we need, and we would not be occupying redundant space on the network.

7.2 Supernetting

As you may imagine, supernetting is the opposite of subnetting.

Grouping smaller networks into a bigger one.

Let’s assume that we need an address block for a network composed of 1000 hosts.

A class C network provides only 254 addresses, so it is not enough.

We could ask for a class B address, but we would drop 2¹⁶ — 2–1000 = 64534 addresses.

So, the solution would be using multiple class C networks.

A network with 1000 hosts needs at least 4 classes C.

Example:

200.45.64.0 → 11001000 . 00101101 . 010000 00 . 00000000

200.45.65.0 → 11001000 . 00101101 . 010000 01 . 00000000

200.45.66.0 → 11001000 . 00101101 . 010000 10 . 00000000

200.45.67.0 → 11001000 . 00101101 . 010000 11 . 00000000

See that we need at least 10 bits to differentiate the networks, which brings us to a /22 mask to group them all into the same bigger network.

Finally, it is worth to remark that Internet routers use this idea of classless routing, and so fort, Internet routers abandoned the old scheme of old IP classes.

8. CIDR

Classless Inter-Domain Routing (CIDR) means that Internet routers do not follow the classes scheme.

It implies that we can aggregate many networks and route them with just one entry to be effective.

CIDR is suitable for reducing the growth of routing tables and for providing efficient routing in the backbone Internet routers.

It reduces the number of table entries as we can aggregate many blocks of addresses and route according to these supernets.

With few entries, the processing time due routing is decreased.

After this idea came into play, classes made no sense anymore inside the Internet, and nowadays the routing tables of the Internet routers are also classless.

However CIDR, supernetting and subnetting are all compatible.

Their combined usage provides effective management of address allocation and routing tables.

• Edge routers use long prefixes for their subnets and short prefixes to describe other networks.
• Core routers in the Internet backbone try to route packets with the shortest prefix possible (biggest possible supernet).

We are getting to the end of this article. I hope you found what you were looking for, and wish that you now have a better understanding of the Internet Protocol, and how do we connect and send packets between devices, even if they share the same network, or if they are in another part of the world.

This is the second episode of a complete series that will guide you through the Internet Basics, so in case you want to improve your digital knowledge, and have a solid understanding of how does the internet work, keep tuned in to my profile as I’ll keep posting articles about this topics.

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

Hello dear reader! And welcome to my Internet Educational Series.

Here you’ll learn all the basics from the Internet, how does it work, why does it work, the protocols used, and many many more you’ll get to discover as you read my articles.

I hope you like them, and enjoy them as much as I’m doing when writing them.

All right, let’s get into it!

I’ll start from the beginning, trying to cover all aspects of how the Internet evolved to reach what it is nowadays.

1. History

The internet history itself starts a long time ago, with an invention called ARPANET with military purposes as many things have done during history, however, my articles won’t cover this part, they are solely focused on the actual and “true” internet as we know it.

As you may imagine, the internet did not start as the wireless network it is nowadays. At the start, communications were made by cable (most of them still are), and the first technology that was invented aimed to propagate information from one place to another, was called ETHERNET.

The first Ethernet was designed in 1973 by Bob Metcalfe in Xerox Corporation’s Palo Alto laboratory.

This first version was able to operate at 3Mbps over a shared coaxial cable using CSMA-CD.

Metcalfe convinced DEC, Intel and Xerox (“DIX”, Digital Intel and Xerox) to work together to promote Ethernet as a standard. The first DIX standard draft was published on September 30, 1980 by the Institute of Electrical and Electronics Engineers (IEEE) under the standard 802.3.

The DIX standard specified the MAC layer of a 10Mbps local area network called Ethernet.

2. The Ethernet Frame

Let’s see the general design of IEEE 802.3/DIX frame:

• The medium is a coaxial cable (a single link bus)
• A Manchester line code is used to transmit each symbol.
• The transmission of frames is asynchronous.
• Multiple stations are allowed to be connected to the link (multi-point) and the MA (medium access) is based on CSMA-CD.
• Each station is identified by a MAC address of 6 bytes (48 bits).
• There is a 2-byte (16 bits) field for multiplexing MAC users.

As mentioned, the transmission over the coaxial was asynchronous and used the Manchester line coding, thus the preamble and SDF were necessary to synchronize the receiver.

• Mac Addresses
• Mac addresses are formed by 48 bits, and it serves as Hardware address, physical address, Ethernet address or layer 2 address.
• It is usually expressed in Hexadecimal (12:34:56:78:9A:BC)
• The standard transmission order sends the Most Significant Byte (MSB) first, and inside this byte the least significant bit (lsb) first.
• For example, the MSB of 12:34:56:78:9A:BC is 12. Which in binary, 12 is 00010010, and we start transmitting the lsb: 01001000…
• The meaning of the lsb and MSB is:
• 0: individual address (unicast)
• 1: Group address (Multicast) I’ll explain more in depth what is Multicast in a future article
• If we find that all 48 bits are 1 (FF:FF:FF:FF:FF:FF) it means that we are transmitting to broadcast.
• Mac User Data

The maximum user data that can be carried in a L2 frame is called the Maximum Transfer Unit (MTU).

For the Ethernet, the MTU is 1500 bytes. The minimum quantity of bytes of user data is 46 bytes. In case our data is less than 46 bytes, we have to pad the data by adding zeros at the end (zero padding).

Notice that the MAC user must include a length field in its PDU to distinguish its data from the padding.

• Frame Check

This field is a Cyclical Redundancy Check (CRC) of 4 bytes (32 bits).

The CRC is used to detect incorrect frames.

The CRC value calculated by the destination, might not match the FCS field value f the frame for 2 reasons:

• Errors
• Collisions

It is important to know that 802.3 receivers DO NOT implement error control, but erroneous frames are silently discarded.

In case we want to manage errors at layer 2, we have to use the optional LLC sublayer, which is defined in the standard IEEE 802.2 (We’ll se more about LLC below).

The MAC layer of the Ethernet MUST be the one who manage collisions.

3. How do we connect senders & receivers? With a Twisted Pair

In the above pictures we can see how two stations are connected via Ethernet with twisted pairs crossover cables.

But what to do with more stations? We use HUBS.

Let’s see some examples to understand better how to connect multiple stations:

3 stations:

As we can see, the TX cable from the 1st station is connected to RX entries from the other ones.

And we can even connect hubs with other hubs!

This way, when hubs are connected between them, we do just have to crossover the cables, and we can send information to every station on the network.

However, when transmitting with hubs, collisions appear, and this led engineers to think of a better way to connect stations between them.

4. Full-Duplex Ethernet

The BRIDGE concept

When trying to connect multiple stations, many collisions were found, and to try to avoid them, engineers came with an idea.

The idea was to separate collision domains.

However, this broke the concept of having one MAC per link, thus developers had to develop a data link layer network capable of managing different links.

Let’s see some important features from the Bridges:

• A bridge is not a physical device like the hub. The bridge processes the frames, and analyses the destination address of each frame.
• If the destination MAC address belongs to the same collision domain of the incoming port, the bridge drops the frame.
• If the destination MAC address belongs to the other collision domains, the bridge sends the frame through the other port.

But how does the bridge know which MAC address belongs to each collision domain?

The answer is called MAC LEARNING, which is an algorithm used by the Ethernet.

The bridge learns mapping between MAC addresses and collision domains by observing the source address of each received frame.

Let’s see an example:

• When A sent the frame, the bridge learned that the MAC of A is in the west collision domain.
• Later when B sends a frame to A, this frame is not transmitted to the East port because the bridge knows that the MAC of A is in the west domain.
• Notice that when B sends the frame to A, the bridge learns that B is also in the west collision domain.

The MAC learning algorithm runs as an uncoordinated process in each bridge, in the sense that the algorithm does not require any coordination with other network devices.

This makes it not scalable for big networks, as a global coordinated process is necessary for such networks.

The Switch Concept

When we extend the bridge concept to a device with more than two ports, we arrive to the concept of SWITCHING.

Simply stated, a switch is a multi-port bridge.

When the switch receives a frame with an unknown MAC, it maps the MAC to the port and sends the frame for all its ports except through the incoming port.

When a MAC is learned, the frame is sent only to the associated port.

Let’s see what happens when D responds to A sending a new frame:

Implementation of a Switch

In this section I’ll describe a possible simplified implementation of a switch with 4 ports full-duplex of 10Mbps:

• Queues are used in each port for buffering incoming and outgoing frames.
• The bus we need has to be at least a shared bus of 40 Mbps.
• This bus is called a ”back plane”.
• We manage the back plane bus with TDM deterministic multiplexing using four time slots.
• Using the shared bus, each port is switched with another port (or ports) at a rate of 40 Mbps but only gets 1 slot each 4 time slots (which yields the 10Mbps transmission rate).
• Notice that with this design, finally we obtain a switch of 4 ports without collisions, at a rate of 10Mbps, that can be used for a full-duplex communication.

Data Link Layer Networks

The next step is to build a “data link layer network” by interconnecting several switches.

We can see that the same algorithm is applied to the data link layer network.

Once all the MAC addresses have been learned, broadcasting is not used anymore and the frames are forwarded using the MAC address table.

This way there are no collisions because the frames are always transmitted “alone”.

5. Broadcast Effects

Broadcast as you may know is the act of sending a frame to EVERYONE in the network. Not only to all switches but to all stations connected to the switches.

The broadcast MAC address is FF:FF:FF:FF:FF.

When a switch receives a frame with a broadcast address, it sends the frame through all its port except through the incoming one, idem as in MAC learning.

However, broadcast should not be overused as it can cause buffer overflows. It is a limiting factor because it generates traffic in the reception of all the ports of the switch.

6. Redundancy

One of the limitations of the MAC learning algorithm is that it generates transmission loops in redundant physical topologies.

A redundant topology provides several paths for connecting a source with a destination. It might be interesting for providing a reliable switching service; in case a path is unavailable, we can use another one.

Spanning Trees

A solution for fixing L2 switching loops is to generate a logical tree. These logical trees are called “spanning trees” and they avoid possible switching loops.

A protocol called Spanning Tree Protocol (STP) is used to dynamically build the spanning tree.

If switching path becomes unavailable, the STP dynamically generates another spanning tree.

7. Final Remarks

Modern switches use several physical signals to auto-negotiate certain aspects of the communication.

• A physical signal allows a station to automatically detect if the device is a switch or a hub.
• In connections between switches, physical signals allow switches to detect each other and negotiate which one is going to be the null modem port.
• Stations and switches also negotiate transmission rates.
• The standard of Ethernet at 100Mbps is called “Fast Ethernet”.
• In Fast Ethernet a pair of twisted wires is used for each direction.

Well, we are getting to the end of this article. I hope you found what you were looking for, and wish that you now have a better understanding of Ethernet protocols, and how do we connect networks using switches.

This is just the first episode of a complete series that will guide you through the Internet Basics, so in case you want to improve your digital knowledge, and have a solid understanding of how does the internet work, keep tuned in to my profile as I’ll keep posting articles about this topics.

Leave a clap if you enjoyed this post, you’ll support my work and help me keep myself motivated to write more! Knowledge is power!

Dear reader, before you read this article I want you to know this is not financial advice. Do your own research before investing your money. This is an educational intended article. I’m just explaining what I wish someone had told me before I started the Cryptocurrencies journey.

I guess if you’ve landed over here you do already know what Bitcoin is, but you do not fully get how does it work, so stop worrying, you are in the right place!

First of all I’ll make an introduction about Blockchain Technology, and later on I’ll explain how does Bitcoin implement this Technology.

Blockchain is nothing more than a specific type of database (I assume you do know what a database is), but its main difference with traditional databases is the way it stores and structures data.

A Blockchain, stores its data in groups, also known as blocks. These blocks do have a limit of storage, and when they are filled, they get chained onto the previous block, forming a chain known as a blockchain. Each one of this blocks, when it joins the blockchain, is given an exact timestamp.

The interesting part about this specific type of databases is that it operates inside a network of computers. Before each block is chained, it needs to be approved by the majority of the computers that form the network, this way the network can prevent malicious information being added to the chain.

If we talk about Bitcoin, each block stores transactions which go from point A to point B. That is basically Bitcoin, a public network which stores a database of transactions.

Regarding other cryptocurrencies, their blockchains can store things like legal contracts, inventories and others, but Bitcoin stores transactions.

But let’s dive a little deeper into Bitcoin’s blockchain.

The purpose of Bitcoin is to provide a Decentralized, Transparent and Trustworthy way of transferring and storing money.

Let’s start with the decentralization part:

As I said, Bitcoin is just a specific type of database that stores every Bitcoin transaction ever made. This database is distributed between a large set of computers, and these computers aren’t usually managed by the same person or group of people. This way, we do have the blockchain distributed in different geographic locations around the world. And the fun part is that as the Bitcoin protocol is open source, ANYONE who wants to contribute to the Bitcoin blockchain can do it just by running their own node in their computer!

Apart from contributing to the network, if you run a node you can have your own wallet there, which is kinda cool!

A node is just another point of the network, where all the network transactions get stored and verified. If one node has an error in its data it can use the thousands of other nodes as a reference point to correct itself. This way, no one node within the network can alter information held within it. Because of this, the history of transactions in each block that make up Bitcoin’s blockchain is irreversible.

If one user want to change the data stored in one transaction, all other nodes will just automatically say “Hey! That’s not what it says in my data”, and the malicious activity will get discarded.

This ensures that the whole blockchain acts in the interest of the majority.

Once you understand the decentralized nature of Bitcoin , it’s much easier to understand its transparency. Each node stores a copy of all the information that has ever gone through the blockchain, so anyone running a node can see the source and destination of the transactions, and this way they can be easily tracked.

Also you can make use of websites that track blockchain transactions in case you do not run a node.

This brings us to question ourselves why governments are talking about Bitcoin being the “criminals money”, if transactions can be easily tracked, but we won’t cover this topic in this article.

Alright, we are heading to the last part. Why is Blockchain Technology secure?

Let’s think on how does the blockchain add their blocks to the chain.

New blocks are stored linearly and chronologically, so they are added at the “end” of the chain. This means that it’s really difficult to go back to a previous block than the last one and alter its data, unless a general consensus is reached.

The difficulty lies in how blocks are stored with a unique identifier. This unique identifier is called a hash.

Hashes are mathematical functions that turns digital information into a random string of numbers and letters. If the information inside the hash changes, the hash itself gets changed too.

Let’s put an example to understand why that’s important to security:

Imagine someone wants to alter the data stored inside a block, to move a transaction destination to their own wallet. Once all the other nodes from the network compare their hash to the one that has been altered, they would see a hash standing out from all the other ones, and the network would automatically discard the transaction that altered it.

The only way to succeed with such a hack would be to control the 51% of the nodes of the network, and here comes a really interesting insight.

Everything can be hacked in some way or another. The only way to prevent something being hacked, is to make that hack a nonsense.

Once someone hacks the 51% of the network, thanks to it being public, the remaining 49% will be able to see it, and as you know, if no one wants the 49% of the network, that network stops having a real value, and so forth, the hacker will own a 51% of a non-valuable network.

Apart, the hacker would have to spend HUGE amounts of time and energy (and so forth, money) to change a single transaction. Here is were we find an explanation on WHY Bitcoin does not use PoS. But that’s another story. In case you are interested in it, contact me and we can discuss further about PoW vs PoS.

Blockchain & Bitcoin history:

Blockchain even if it doesn’t seem so, was invented around 1990s, but it wasn’t until 2009 that it found it’s first real-world application: Bitcoin, created originally by Satoshi Nakamoto, who no one still knows who he is.

And this first blockchain application was designed to work as Satoshi said: “a new electronic cash system that’s fully peer-to-peer, with no trusted third party.”

But as you now know, blockchain can be used in many different ways, not only implementing money system payments, but also election records, logistic procedures, legal transactions, and many more that are yet to be discovered.

Woooah this is getting to an end, and all that information was hard to digest, I know, but I hope you do better understand now how does Blockchain Technology and Bitcoin transactions work.

I hope you liked this article, I just want to say thank you if you reached this far!

Make sure to leave a ‘Clap’ if you enjoyed this article, it supports my work and helps keep myself motivated to write more articles!

Thanks for reading!

Read my other articles!

• What is Bitcoin?
• How to Choose the Best Cryptocurrency Exchange of 2021?