Networking in C · intermediate · ~15 min

TLS — read about it; never write it

Understand TLS structure; integrate via a vetted library.

Overview

TLS (Transport Layer Security) sits between TCP and HTTP. It encrypts and protects the data your application sends.

It works in two stages:

  • Handshake: the two sides agree on a shared secret, using asymmetric cryptography (math where you can share a public key while keeping a private one).
  • Record layer: every chunk of payload is encrypted and given a MAC (a tag that detects tampering).

The golden rule: never implement TLS yourself. Always integrate a vetted library.

Why it matters

TLS is everywhere:

  • Every HTTPS connection uses it.
  • Every modern protocol assumes it.

To read a C program that uses TLS, you need to recognize the common OpenSSL idioms. That is the goal of this lesson.

Core concepts

Record layer. Each record has a 5-byte header (type, version, length) followed by the encrypted payload. The type says what the record carries: handshake, application data, or alert.

Handshake. The opening exchange that sets up the secure channel:

  1. ClientHello
  2. ServerHello
  3. Certificate + key exchange
  4. Finished

TLS 1.3 collapses this into a single round trip (1-RTT).

Certificate verification. In OpenSSL, verification is OFF by default for clients. Always turn it on. Always validate the hostname too.

SNI (Server Name Indication). The client tells the server which hostname it wants during the handshake. This lets one IP address host many certificates.

Pentester mindset. Misconfigured TLS shows up as mixed content, downgrade attacks, expired certificates, or weak ciphers. Defenders audit for these with tools like testssl.sh, sslyze, and nmap --script ssl-enum-ciphers.

Defensive coding habit. A safe client configuration:

  • SSL_VERIFY_PEER
  • a real CA bundle
  • hostname validation
  • a minimum of TLS 1.2

Refuse anything weaker.

Syntax notes

Reference: https://www.openssl.org/docs/man3.0/man7/ssl.html.

Note OpenSSL's unusual convention: most functions return 1 on success.

Lesson

TLS is the layer between TCP and your application. It provides three things:

  • encryption
  • authentication
  • integrity

You will never write a TLS stack from scratch. Instead, you wrap a vetted library such as OpenSSL, BoringSSL, mbedTLS, or GnuTLS.

This lesson explains the structure of TLS so you can read someone else's TLS code with confidence.

Code examples

SSL_CTX *ctx = SSL_CTX_new(TLS_client_method());
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
SSL_CTX_load_verify_locations(ctx, NULL, "/etc/ssl/certs");
SSL *ssl = SSL_new(ctx);
SSL_set_fd(ssl, tcp_fd);
SSL_connect(ssl);
SSL_write(ssl, msg, mlen);

Line by line

See the OpenSSL wiki.openssl.org 'Simple_TLS_Server' / 'Simple_TLS_Client' pages.

Common mistakes

  • Skipping certificate verification. The classic excuse is "I'll add it later" — and later never comes. Without verification, the encrypted connection gives you no guarantee about who is on the other end.

Debugging tips

  • openssl s_client -connect example.com:443 -showcerts is the gold-standard troubleshooting tool. It shows the full handshake and certificate chain.
  • tcpdump captures the record layer on the wire for deeper analysis.

Memory safety

OpenSSL has its own cleanup rules:

  • Always call SSL_free and SSL_CTX_free to release objects.
  • Always check return values against the 1 / 0 / -1 contract.

Real-world uses

  • curl
  • Every HTTPS client and server
  • Every modern TCP protocol that needs authentication plus encryption

Practice tasks

  1. Read the OpenSSL Simple_TLS_Client example from start to finish.
  2. Explain in your own words what SSL_VERIFY_PEER does.
  3. Run testssl.sh example.com and read through the report.

Summary

  • Do not implement TLS yourself — integrate a vetted library.
  • Verify certificates and validate the hostname.
  • Require TLS 1.2 or higher; refuse weak protocols and ciphers.
  • Always free OpenSSL objects and check return values.

Practice with these exercises