# ZeroMQ Chapter 2 Deep Notes: Sockets and Patterns

Source material: `stripped_zeromq-messaging-for-many-applications_compress.pdf`, Chapter 2, "Sockets and Patterns".

These notes explain Chapter 2 as a practical guide. Each major topic includes what it means, why it exists, how it works, an easy example, common mistakes, and takeaways.

## 1. Main Idea of Chapter 2

Chapter 1 introduced the first patterns. Chapter 2 explains how to use ZeroMQ sockets correctly in real programs.

The central idea is:

> A ZeroMQ socket is a typed message-pattern endpoint, not a plain TCP connection.

The socket type decides:

- which peer socket types are valid
- whether the socket sends, receives, or both
- whether it load balances, broadcasts, routes, or queues
- what happens when queues fill

This means you cannot understand a ZeroMQ program by only looking at `send()` and `recv()`. You must also understand the socket types and the topology.

## 2. Socket Lifecycle

### What it means

A ZeroMQ socket goes through a lifecycle:

1. Create it.
2. Configure it.
3. Attach it to the topology.
4. Send or receive messages.
5. Close it.

In C:

void *socket = zmq_socket(context, ZMQ_REQ);
zmq_setsockopt(socket, option, value, value_size);
zmq_connect(socket, "tcp://localhost:5555");
zmq_msg_send(&message, socket, 0);
zmq_close(socket);

### Why it exists

This lifecycle separates responsibilities:

- `zmq_socket()` chooses behavior.
- `zmq_setsockopt()` adjusts details.
- `zmq_bind()` or `zmq_connect()` places the socket in the network.
- `zmq_msg_send()` and `zmq_msg_recv()` move messages.
- `zmq_close()` releases the socket.

### Easy example

Think of opening a service counter:

1. Choose counter type: information desk, payment counter, pickup counter.
2. Configure rules: open hours, queue limit.
3. Put the counter at a known location.
4. Serve people.
5. Close the counter.

The counter type matters. A payment counter and pickup counter are both "counters," but they have different rules. ZeroMQ sockets work the same way.

### Common mistake

Creating a socket type without understanding its pattern. For example, using `PUB` when you need replies, or using `REQ` when you need multiple outstanding requests.

### Practical takeaway

Always write down the socket type and expected peer type before coding a ZeroMQ flow.

## 3. Bind and Connect

### What it means

To connect two ZeroMQ sockets, one side binds to an endpoint and the other connects to that endpoint.

zmq_bind(server, "tcp://*:5555");
zmq_connect(client, "tcp://localhost:5555");

`bind()` says: "I provide this endpoint."
`connect()` says: "I attach to that endpoint."

### Why it exists

Distributed systems need a topology: which components know about which addresses. The bind/connect choice controls how easy it is to add and remove nodes.

Useful rule:

- stable components bind
- dynamic components connect

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    Stable["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>Stable service<br/>binds fixed endpoint</div>"]
    D1["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Dynamic node<br/>connects</div>"]
    D2["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Dynamic node<br/>connects</div>"]
    D3["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Dynamic node<br/>connects</div>"]
    D1 --> Stable
    D2 --> Stable
    D3 --> Stable

### How it differs from TCP

In raw TCP server code, a server usually calls `bind()`, `listen()`, and `accept()`. ZeroMQ does not expose `accept()` to the application. Once a ZeroMQ socket binds, ZeroMQ accepts connections internally.

Also, ZeroMQ connections can be established asynchronously. A client can call `zmq_connect()` before the server exists. ZeroMQ may queue or reconnect depending on socket type and state.

### Easy example

A classroom has one fixed classroom number. Students come and go.

- Classroom = stable component, so it binds.
- Students = dynamic components, so they connect.

If every student created a different classroom number, the teacher would need to track all of them. That is harder to scale.

### Common mistake

Making dynamic workers bind. Then every new worker endpoint must be added to the ventilator or broker configuration.

### Practical takeaway

Choose bind/connect based on topology stability, not just "server versus client" names.

## 4. Transports

### What it means

ZeroMQ can use different transports while keeping the same socket API.

| Transport | Use case | Example endpoint |
| --- | --- | --- |
| `tcp` | network communication | `tcp://host:5555` |
| `ipc` | processes on same machine | `ipc://weather.ipc` |
| `inproc` | threads in same process | `inproc://workers` |
| `pgm`, `epgm` | multicast | advanced cases |

### Why it exists

The same messaging pattern might start in one process and later move across machines. ZeroMQ lets you change transport without rewriting the message logic.

### Important details

- `tcp` is the normal choice for examples and most networked programs.
- `ipc` is useful for local processes, but not portable to every operating system.
- `inproc` is very fast because it stays inside one process.
- `inproc` requires the bind side to happen before the connect side.
- multicast transports are specialized and should not be the first choice.

### Easy example

The same conversation can happen:

- across a table: `inproc`
- between rooms in the same building: `ipc`
- between buildings: `tcp`

The message can be the same, but the delivery path changes.

### Practical takeaway

Start with `tcp` for normal distributed examples. Use `inproc` for thread communication inside one process.

## 5. ZeroMQ Framing Versus Raw Protocols

### What it means

ZeroMQ puts its own frame format on the wire. This makes message boundaries clear, but it also means ZeroMQ sockets do not normally speak existing protocols such as HTTP directly.

ZeroMQ Framing
:::info
[LENGTH byte][FLAGS byte][actual data bytes...]
[LENGTH byte][FLAGS byte][actual data bytes...]
:::

HTTP over TCP has HTTP-specific text framing. ZeroMQ over TCP has ZeroMQ framing.

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    HTTP["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>HTTP over TCP<br/>text headers<br/>CRLF rules</div>"]
    ZMTP["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>ZeroMQ over TCP<br/>length frames<br/>message pattern</div>"]
    TCP["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>TCP<br/>byte transport</div>"]
    HTTP --> TCP
    ZMTP --> TCP

### Why it exists

Framing solves a basic problem: the receiver must know how much data belongs to one message. ZeroMQ solves this for ZeroMQ programs by adding length-specified frames.

### Common mistake

Trying to write an HTTP server just by replacing raw TCP sockets with ordinary ZeroMQ sockets. The wire format will not match HTTP.

### Practical takeaway

Use ZeroMQ when both sides are ZeroMQ-aware, or build a bridge that translates between protocols.

## 6. I/O Threads and Background Queues

### What it means

ZeroMQ does network I/O in background threads owned by the context. When your application calls `zmq_msg_send()`, the message is usually queued locally and sent later by the I/O thread.

### Why it exists

This design lets application code stay focused on messages while ZeroMQ handles:

- connection management
- reconnection
- buffering
- outgoing network writes
- incoming network reads

### How it works conceptually

application thread -> local ZeroMQ queue -> I/O thread -> network
network -> I/O thread -> local ZeroMQ queue -> application thread

### Easy example

Imagine a post office counter. You hand a letter to the counter clerk. The letter is accepted immediately, but the truck picks it up later. Your call to `send()` does not always mean the message has already reached the receiver.

### Common mistake

Assuming `zmq_msg_send()` means "remote peer has received this." It usually means "ZeroMQ accepted this message for sending."

### Practical takeaway

Think in terms of queues and asynchronous delivery. This helps explain slow joiner behavior, HWM, and shutdown issues.

### Q and A part

**Q: When you call zmq_send(), is the message sent immediately?**

A: No. The message is **queued locally** and sent later by a background I/O thread.

**Sequence:**
1. Your code calls `zmq_send(socket, "hello")`
2. Message goes to **outgoing message queue**
3. Function returns immediately to your code
4. Background I/O thread sends from the queue asynchronously

**Q: Where is the message stored?**

A: In the **outgoing message queue** (also called the send buffer). ZeroMQ maintains one queue per connection.

**Q: Who sends the message over the network?**

A: The **I/O thread** owned by the context, running in the background.

**Q: What is the High Water Mark (HWM)?**

A: The **maximum number of messages** that can queue before ZeroMQ blocks or drops messages.

socket.setsockopt(zmq.SNDHWM, 100)  # Max 100 messages in queue

**Q: Is ZeroMQ considered a high-level framework?**

A: Yes, because **you don't have to manage:**
- Creating or managing threads
- Creating or managing queues
- Sending data over raw sockets
- Handling reconnections
- Buffering or serialization details
- Connection state management

## 7. Valid Socket Pattern Combinations

### What it means

ZeroMQ socket types are not arbitrary. Each socket belongs to a pattern.

Useful combinations include:

- `REQ` with `REP`
- `REQ` with `ROUTER`
- `DEALER` with `REP`
- `DEALER` with `ROUTER`
- `DEALER` with `DEALER`
- `ROUTER` with `ROUTER`
- `PUB` with `SUB`
- `PUSH` with `PULL`
- `PAIR` with `PAIR`

### Why it exists

Socket type defines routing behavior. If the types do not match, the rules do not make sense.

For example:

- `PUB` broadcasts and does not receive.
- `SUB` receives and does not send.
- `PUSH` distributes work.
- `PULL` receives work.

Connecting `PULL` to `SUB` has no meaningful pattern.

### Common mistake

Trying a random socket pair because it compiles. ZeroMQ programs may compile but still have undefined or unreliable messaging behavior.

### Practical takeaway

Before writing code, write the pattern in words:

clients REQ -> broker ROUTER -> broker DEALER -> workers REP

If the pattern is not valid, fix the design first.

## 8. Working with `zmq_msg_t`

### What it means

In C, ZeroMQ messages are handled with `zmq_msg_t`. This object represents a message frame.

Receive pattern:

zmq_msg_t message;
zmq_msg_init(&message);
zmq_msg_recv(&message, socket, 0);
// inspect message
zmq_msg_close(&message);

Send pattern:

zmq_msg_t message;
zmq_msg_init_size(&message, size);
memcpy(zmq_msg_data(&message), data, size);
zmq_msg_send(&message, socket, 0);
zmq_msg_close(&message);

### Why it exists

`zmq_msg_t` lets ZeroMQ manage message memory efficiently. It can reference-count or move message data internally.

### Easy example

Think of `zmq_msg_t` as a package container. You put data into the container and hand the container to ZeroMQ. After sending, do not keep assuming the container still holds your original package in the same way.

### Common mistake

Using message data after `zmq_msg_send()` as if nothing changed. After sending, the message object is no longer a normal application-owned buffer.

### Practical takeaway

Create, send or receive, then close. Keep message ownership simple.

## 9. Handling Multiple Sockets with `zmq_poll()`

### What it means

`zmq_poll()` lets one thread wait for activity on multiple sockets.

Without polling, a program that blocks on one socket may miss work on another.

### Easy example: receptionist watching two doors

Imagine a receptionist who must watch:

- front door: customers arrive
- side door: delivery packages arrive

If the receptionist stares only at the front door, packages may wait forever. `zmq_poll()` is like watching both doors and responding to whichever one has activity.

### Basic flow

zmq_pollitem_t items[] = {
    { receiver, 0, ZMQ_POLLIN, 0 },
    { subscriber, 0, ZMQ_POLLIN, 0 }
};

zmq_poll(items, 2, -1);

if (items[0].revents & ZMQ_POLLIN) {
    // receive from receiver
}

if (items[1].revents & ZMQ_POLLIN) {
    // receive from subscriber
}

### Common mistake

Using repeated nonblocking receive plus sleep as the main design. That works as a demonstration, but adds latency and wastes CPU compared with polling.

### Practical takeaway

Use `zmq_poll()` when one thread owns multiple sockets.

## 10. Multipart Messages

### What it means

A multipart message is one logical message made of several frames.

Examples:

- frame 1: address
- frame 2: empty delimiter
- frame 3: body

or:

- frame 1: topic
- frame 2: payload

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    E["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Frame 1<br/>envelope</div>"]
    M["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Frame 2<br/>metadata</div>"]
    B["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Frame 3<br/>body</div>"]
    E -->|"SNDMORE"| M
    M -->|"SNDMORE"| B
    B -->|"final"| Done["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Complete<br/>multipart message</div>"]

### Why it exists

Multipart messages let applications keep related pieces separate without manually combining them into one buffer.

This is useful for:

- routing envelopes
- topic keys
- metadata plus body
- forwarding messages without understanding their content

### How to send

zmq_msg_send(&part1, socket, ZMQ_SNDMORE);
zmq_msg_send(&part2, socket, ZMQ_SNDMORE);
zmq_msg_send(&part3, socket, 0);

### How to receive

Receive one frame at a time and check `ZMQ_RCVMORE`.

### Easy example: envelope and letter body

A physical letter has:

- envelope: tells where it should go
- paper inside: message content

You should not glue them together if the postal system needs the envelope separately.

### Common mistake

Forwarding only the first frame of a multipart message. This breaks routing envelopes and causes replies to disappear.

### Practical takeaway

When writing a proxy, forward every frame until `ZMQ_RCVMORE` says there are no more.

### Q and A Session

Q: If a message contains multiple frames, are all those frames sent together as one unit, or separately?
A: All frames are sent together as one atomic unit.

Q: When the receiver gets a multipart message, does it receive all frames at once, or does it have to ask for them one by one?
A: The receiver gets all frames, but must ask for them one by one using zmq_recv().

Q: Look back at the frame structure we learned: Each frame has a FLAGS field. What do you think that flags field might be used for in a multipart message?
A: The FLAGS field contains control bits, especially the "MORE" flag that tells the receiver if more frames are coming.

## 11. Intermediaries and Proxies (SAMPE SINI)

### What it means

An intermediary is a middle component that receives messages on one side and forwards them to another side.

Examples:

- pub-sub forwarder
- request-reply broker
- bridge between networks
- shared queue

### Why it exists

Without intermediaries, every node may need to know every other node. That makes the topology hard to change.

With an intermediary:

- clients know the broker
- workers know the broker
- clients do not need to know workers
- workers can be added without touching clients

### Easy example

A front desk in an office receives visitors and sends them to available staff. Visitors do not need every staff member's office number. Staff do not need to know every visitor in advance.

### `zmq_proxy()`

ZeroMQ provides `zmq_proxy(frontend, backend, capture)` for common forwarding loops. The sockets still must be created, bound or connected, and configured correctly.

### Common mistake

Putting too much business logic in the proxy. A proxy should often be a simple message switch unless the design intentionally requires more.

### Practical takeaway

Use proxies to reduce topology complexity, but keep them simple.

### Dynamic Discovery Problem

## 12. Shared Queue with ROUTER and DEALER

### What it means

A shared queue connects many clients to many workers.

Common structure:

- clients: `REQ`
- broker frontend: `ROUTER`
- broker backend: `DEALER`
- workers: `REP`

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    C1["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>REQ<br/>client A</div>"]
    C2["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>REQ<br/>client B</div>"]
    R["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>ROUTER<br/>frontend</div>"]
    D["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>DEALER<br/>backend</div>"]
    W1["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>REP<br/>worker 1</div>"]
    W2["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>REP<br/>worker 2</div>"]
    C1 --> R
    C2 --> R
    R <--> D
    D --> W1
    D --> W2

### Why `REQ/REP` alone is not enough

Plain `REQ/REP` is strict. A broker needs to accept messages from either side whenever they arrive. That requires asynchronous behavior, which `ROUTER` and `DEALER` provide.

### Easy example: front desk assigning work

Customers do not walk directly into staff offices. They go to the front desk. The front desk assigns each request to an available worker and later sends the answer back to the correct customer.

### Common mistake

Forwarding request bodies but dropping routing frames. In request-reply brokers, the invisible-looking frames may be the information needed to route replies back.

### Practical takeaway

When proxying `ROUTER/DEALER`, treat multipart messages as opaque. Forward all frames.


## 13. DEALER and ROUTER Roles

### ROUTER

`ROUTER` is identity-aware. It can know which peer a message came from and route a message back to that peer.

### DEALER

`DEALER` is asynchronous and load-distributing. It does not force the strict send/receive alternation of `REQ`.

:::info
DEALER receives all three requests, BUT does DEALER remember which client sent which request?

DEALER just say "I got 3 requests, but I don't know which client they came from"?
:::

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    Router{"<div style='min-width:130px;line-height:1.3;padding:4px 6px'>ROUTER<br/>tracks peer identity</div>"}
    Dealer{"<div style='min-width:130px;line-height:1.3;padding:4px 6px'>DEALER<br/>async distribution</div>"}
    Router -->|"requests"| Dealer
    Dealer -->|"replies"| Router

### Why they matter

They are the building blocks for scalable request-reply topologies. They let a system handle many clients and many workers without every client knowing every worker.

### Practical takeaway

`REQ/REP` is simple. `ROUTER/DEALER` is flexible. Use the simple pattern first, then move to `ROUTER/DEALER` when topology requires it.

## 14. Error Handling: `EAGAIN`, `ETERM`, and `EINTR`

### What it means

Real C programs should check return values from ZeroMQ calls.

Common patterns:

- object creation returns `NULL` on failure
- many operations return `-1` on failure
- error detail is in `errno` or `zmq_errno()`

### `EAGAIN`

Occurs when a nonblocking operation cannot proceed now.

Example:

receive with ZMQ_DONTWAIT
no message is ready
return -1, errno = EAGAIN

This is not necessarily fatal. It often means "try later."

### `ETERM`

Occurs when the ZeroMQ context is terminated while another thread is blocked in a ZeroMQ call.

This is part of shutdown behavior. Threads should treat it as a signal to clean up and exit.

### `EINTR`

Occurs when a blocking call is interrupted by a signal such as Ctrl-C.

### Common mistake

Treating all `-1` returns as identical. Some are fatal bugs. Some are normal control-flow events.

### Practical takeaway

Check `errno` and decide whether the error is fatal, retryable, or part of shutdown.

## 15. Interrupt Handling

### What it means

Programs should handle Ctrl-C and termination signals cleanly.

Pattern:

1. Install a signal handler.
2. Handler sets a global interrupted flag.
3. Main loop checks the flag.
4. Program closes sockets and destroys context before exiting.

### Easy example

If a classroom fire alarm rings, students should not vanish instantly. They should stop work, close notebooks, and exit in an orderly way. A signal handler gives the program a chance to do the equivalent cleanup.

### Common mistake

Catching Ctrl-C but never checking the flag. Then the program appears impossible to interrupt normally.

### Practical takeaway

Signal handlers should be simple. Set a flag, then let normal code clean up.

## 16. Memory Leak Checking

### What it means

In C and C++, the application must release messages, sockets, and contexts correctly.

Tools such as `valgrind` help detect leaks.

### Why clean exit matters

If the program is killed abruptly, memory tools may report resources that were still allocated. That does not always mean the program leaked during normal operation. To get useful reports, make the program exit cleanly.

### Practical takeaway

Long-running ZeroMQ programs need both:

- correct cleanup code
- a clean path to reach that cleanup code

## 17. Multithreading with ZeroMQ

### What it means

ZeroMQ encourages message passing between threads instead of shared-state concurrency.

Rules:

- share the context if threads use `inproc`
- do not share sockets between threads
- each thread creates and closes its own sockets
- communicate using messages

### Why it exists

Shared-state multithreading often needs locks, mutexes, and careful ordering. That is hard to reason about. Message passing reduces accidental shared state.

### Easy example

Instead of two people editing the same paper at the same time, each person writes a note and sends it to the other. The paper is not shared directly; messages coordinate the work.

### Multithreaded server pattern

Typical structure:

- main thread binds `ROUTER` for external clients
- main thread binds `DEALER` for internal workers using `inproc`
- worker threads create `REP` sockets and connect to the internal endpoint
- `zmq_proxy()` moves messages between clients and workers

### Common mistake

Creating a socket in one thread and using or closing it in another. This may appear to work and later fail randomly.

### Practical takeaway

Context may be shared. Sockets should not be shared.

:::info
Without ZeroMQ (Shared-State with Locks):

Thread A: Waiting for lock... waiting... waiting...
Thread B: Waiting for lock... waiting... waiting...
Result: Slow! Only one thread works at a time

With ZeroMQ (Message Passing):

Thread A: Edits its own paper ✓ (no waiting!)
Thread B: Edits its own paper ✓ (no waiting!)
Both: Send messages to each other ✓ (merge information)

:::

## 18. PAIR Sockets for Thread Signaling

### What it means

`PAIR` sockets provide exclusive one-to-one communication. Chapter 2 uses them for thread readiness signaling.

### Easy example: "READY" message

Thread 1 finishes setup and sends:

READY

Thread 2 waits for that message before continuing.

This is better than guessing with `sleep()`.

### Why not use other socket types?

- `PUSH/PULL`: signals may be load-balanced if extra receivers exist.
- `PUB/SUB`: signal can be lost during subscription setup.
- `DEALER/ROUTER`: adds envelope behavior that is unnecessary for a simple pair.

### Practical takeaway

Use `PAIR` for tightly coupled thread pairs. Do not treat it as a general network coordination tool.

## 19. Node Coordination

### What it means

Coordinating network nodes is different from coordinating threads. Nodes can appear, disappear, restart, or reconnect.

The chapter's synchronized pub-sub example solves the problem of subscribers missing the beginning of a stream.

### How it works

Publisher has two sockets:

- `PUB` for data
- `REP` for synchronization

Subscriber has two sockets:

- `SUB` for data
- `REQ` to announce readiness

Flow:

1. Subscriber connects `SUB` and sets subscription.
2. Subscriber sends readiness request on `REQ`.
3. Publisher waits for all expected readiness requests.
4. Publisher starts sending real data.

### Easy example

A teacher does not start the exam until every student says "ready." The lesson is not to assume that being inside the room means everyone is ready.

### Common mistake

Using `PAIR` for network nodes. `PAIR` is not designed for dynamic reconnecting node sets.

### Practical takeaway

Use explicit synchronization channels when startup timing matters.

## 20. Zero-Copy

### What it means

Zero-copy send lets ZeroMQ send from an application-owned buffer without copying into a separate ZeroMQ buffer first.

In C this uses `zmq_msg_init_data()` plus a cleanup callback.

### Why it exists

For very large messages or high-throughput systems, avoiding copies may improve performance.

### Why beginners should be careful

Zero-copy complicates ownership:

- Who allocated the buffer?
- When is it safe to free?
- What function will ZeroMQ call when done?

### Practical takeaway

Use ordinary message creation first. Consider zero-copy only after measurement shows copying is a real cost.

## 21. Pub-Sub Envelopes

### What it means

A pub-sub envelope separates the topic key from the message body by using multipart messages.

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    T["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Frame 1<br/>topic key</div>"]
    Body["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Frame 2<br/>body</div>"]
    Sub["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>SUB filter<br/>matches topic frame</div>"]
    T --> Body
    Sub -.-> T

### Why it helps

Subscription matching is prefix-based. If the topic and body are mixed into one string, accidental matches become possible. By placing topic and body in separate frames, matching stays clean.

### Easy example

Envelope:

weather

Letter body:

Taipei 28C humidity 70%

The subscriber filters on the envelope, not the whole letter.

### Practical takeaway

Use multipart envelopes when topic and payload are logically separate.

## 22. High-Water Marks

### What it means

High-water mark, or HWM, is a queue limit. It controls how many message parts ZeroMQ will queue internally for a pipe.

### Why it exists

If a sender is fast and a receiver becomes slow, messages collect somewhere:

- receiver buffers
- network buffers
- sender buffers
- ZeroMQ queues

Without limits, memory can grow until the process crashes.

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 50, "padding": 10, "htmlLabels": true, "useMaxWidth": false}, "sequence": {"showSequenceNumbers": false, "useMaxWidth": false}, "themeVariables": {"background": "#ffffff", "primaryColor": "#f8fafc", "primaryTextColor": "#111827", "primaryBorderColor": "#1f4e79", "lineColor": "#374151", "secondaryColor": "#eef6ff", "tertiaryColor": "#f5f5f4", "fontFamily": "Arial, sans-serif", "fontSize": "14px", "clusterBkg": "#f8fafc", "clusterBorder": "#64748b", "edgeLabelBackground": "#ffffff", "noteBkgColor": "#fff7ed", "noteTextColor": "#111827", "noteBorderColor": "#c2410c", "actorBkg": "#f8fafc", "actorBorder": "#1f4e79", "actorTextColor": "#111827", "activationBkgColor": "#e0f2fe", "activationBorderColor": "#1f4e79", "signalColor": "#1f4e79", "signalTextColor": "#111827", "labelBoxBkgColor": "#ffffff", "labelBoxBorderColor": "#64748b", "labelTextColor": "#111827", "stateBkg": "#f8fafc", "stateBorder": "#1f4e79", "mainBkg": "#f8fafc", "mainBorderColor": "#1f4e79"}}}%%
flowchart LR
    Sender["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Fast sender</div>"]
    Queue["<div style='min-width:145px;line-height:1.3;padding:4px 6px'>ZeroMQ queue<br/>limited by HWM</div>"]
    Receiver["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>Slow receiver</div>"]
    Action{"<div style='min-width:130px;line-height:1.3;padding:4px 6px'>Queue full?</div>"}
    Sender --> Queue --> Receiver
    Queue --> Action
    Action -->|"some sockets"| Block["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>block sender</div>"]
    Action -->|"PUB/ROUTER"| Drop["<div style='min-width:120px;line-height:1.3;padding:4px 6px'>drop messages</div>"]

### Easy example: waiting room capacity

A clinic waiting room has 20 chairs. If patients arrive faster than doctors can see them, the room fills. The clinic must either stop accepting people temporarily or send some away. HWM is the queue capacity rule.

### Important detail

HWM counts message parts, not whole logical multipart messages. A two-frame message consumes two queue units.

### Practical takeaway

Set and understand HWM in high-volume systems. It protects memory but changes behavior under pressure.

## 23. Missing Message Problem Solver

When messages disappear, do not guess randomly. Check likely causes:

- Did the subscriber set a subscription?
- Did the publisher send before the subscriber connected?
- Are the socket types compatible?
- Did a proxy forward all multipart frames?
- Did HWM block or drop messages?
- Did the program exit before queued messages were sent?
- Is a socket being shared across threads?
- Are return values and errors being ignored?

### Practical debugging approach

1. Reduce to two nodes.
2. Send one message.
3. Add logging around every send and receive.
4. Check every return value.
5. Add multipart frame counting if a proxy is involved.
6. Add more nodes only after the simple case works.

## 24. Chapter 2 Review Questions

1. Why is a ZeroMQ socket not the same as a TCP connection?
2. When should a component bind instead of connect?
3. Why does `inproc` need bind before connect?
4. Why can ZeroMQ not normally speak raw HTTP?
5. What does `zmq_poll()` solve?
6. Why must a proxy forward all multipart frames?
7. What is the difference between `EAGAIN`, `ETERM`, and `EINTR`?
8. Why should sockets not be shared between threads?
9. When is `PAIR` appropriate?
10. What happens when HWM is reached?

## 25. Chapter 2 Practical Summary

- Socket type defines communication behavior.
- Bind/connect design shapes topology.
- ZeroMQ messages are framed and may be multipart.
- `zmq_poll()` is the normal way to watch multiple sockets.
- Proxies reduce topology complexity.
- `ROUTER/DEALER` enable flexible request-reply brokers.
- Error handling must distinguish normal retry/shutdown conditions from real failures.
- In multithreaded programs, share contexts but not sockets.
- HWM protects memory but can block or drop messages depending on socket type.