Hands-on Node.

js
2012 Pedro Teixeira is version was published on 2012-05-11

is is a Leanpub book, for sale at: http://leanpub.com/hands-on-nodejs Leanpub helps authors to self-publish in-progress ebooks. We call this idea Lean Publishing. To learn more about Lean Publishing, go to: http://leanpub.com/manifesto To learn more about Leanpub, go to: http://leanpub.com

Contents
Credits About this book Anowledgements Introduction Why the sudden, exponential popularity? . . . . . . . . . . . . . . . . . . . . . . . . . . . What does this book cover? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What does this book not cover? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Where will this book lead you? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter Overview i ii iii iv iv v v v v v vi vii

Why? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Starting up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Understanding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii API i Tour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Buers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Event Emier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Timers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Low-level File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii TCP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii UNIX Soets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Datagrams (UDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii i

CONTENTS

ii

Child Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Streaming HTTP Chunked Responses . . . . . . . . . . . . . . . . . . . . . . . . . . viii TLS / SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii HTTPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Making Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Callba Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why? Why the event loop? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Solution 1: Create more call stas . . . . . . . . . . . . . . . . . . . . . . . . . . . Solution 2: Use event callbas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why Javascript? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How I Learned to Stop Fearing and Love Javascript . . . . . . . . . . . . . . . . . . Function Declaration Styles . . . . . . . . . . . . . . . . . . . . . . . . . . . Functions are rst-class objects . . . . . . . . . . . . . . . . . . . . . . . . . JSLint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Handling callbas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Starting up Install Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NPM - Node Paage Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NPM commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
npm ls [filter] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . npm install package[@filters] . . . . . . . . . . . . . . . . . . . . . . . . . . . . npm rm package_name[@version] [package_name[@version] ...] . . . . . . . . . npm view [@] [[.]...] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix ix ix ix 1 1 1 2 3 5 5 7 9 10 11 12 12 14 14 14 15 16 16

CONTENTS

iii 17 17 17 18 18 20 21 21 21 21 21 21 23 23 23 23 23 24 24 24 24 24 24 24 24 24 25 25 26

Understanding Understanding the Node event loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An event-queue processing loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Callbas that will generate events . . . . . . . . . . . . . . . . . . . . . . . . . . . Dont blo! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understanding Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Node resolves a module path . . . . . . . . . . . . . . . . . . . . . . . . . . . Core modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modules with complete or relative path . . . . . . . . . . . . . . . . . . . . As a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . As a directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . As an installed module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . API qui tour Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ild_process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . File system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . fs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . net . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dgram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . tls (ssl) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Utilities console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . util . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

iv 28 29 29 30 30 30 30 31 31 31 32 32 33 33 33 34 34 34 35 35 36 36 38 38 39 40 41 44 44

Buers Slice a buer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Copy a buer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Buer Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Event Emitter .addListener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .once . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .removeAllListeners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating an Event Emier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Event Emier Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Timers setTimeout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . clearTimeout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . setInterval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . clearInterval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . process.nextTi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Escaping the event loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Low-level le-system fs.stat and fs.fstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Open a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Read from a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Write into a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . File-system Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 - get the size of a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

v 44 44 45 45 45 46 46 47 47 47 47 48 48 49 49 49 50 50 50 51 51 51 52 52 53 53 53 54 54

Exercise 2 - read a unk from a le . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 3 - read two unks from a le . . . . . . . . . . . . . . . . . . . . . . . . Exercise 4 - Overwrite a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 5 - append to a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 6 - ange the content of a le . . . . . . . . . . . . . . . . . . . . . . . . . HTTP HTTP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . e hp.ServerRequest object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . req.url . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . req.method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . req.headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . e hp.ServerResponse object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Write a header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Change or set a header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Remove a header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Write a piece of the response body . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hp.get() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hp.request() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Streams, pump and pipe ReadStream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wait for data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Know when it ends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pause it . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

vi 54 54 54 55 55 55 56 56 57 57 58 60 60 61 61 61 62 62 63 63 63 65 65 65 65 66 66 66 67

Resume it . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . WriteStream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wait for it to drain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some stream examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Filesystem streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . e Slow Client Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What can we do? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . and his cousin pipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP Write a string or a buer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . and all the other methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Idle soets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keep-alive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Delay or no delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . server.close() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Listening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UNIX soets Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Passing le descriptors around . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

vii 67 68 70 70 71 73 73 74 74 74 75 75 76 77 77 77 78 78 79 79 80 80 80 80 81 82 82 82

Read or write into that le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Listen to the server soet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Datagrams (UDP) Datagram server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Datagram client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Datagram Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Receiving multicast messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sending multicast messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UDP Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Child processes Executing commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spawning processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Killing processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Child Processes Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Streaming HTTP unked responses A streaming example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Streaming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS / SSL Public / private keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Private key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Public key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

viii 82 82 83 83 83 84 84 84 86 86 86 87 88 89 89 90 91 91 91 92 96 97 97 98 99 99 99 99

Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercise 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTPS HTTPS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTPS Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Making modules CommonJS modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One le module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An aggregating module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A pseudo-class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A pseudo-class that inherits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . node_modules and npm bundle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bundling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Debugging console.log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Node built-in debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Node Inspector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Live edit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated Unit Testing A test runner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assertion testing module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . should.js . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assert truthfulness: . . . . . . . . . . . . . . . . . . . . . . . . . . . or untruthfulness: . . . . . . . . . . . . . . . . . . . . . . . . . . . . === true . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

ix === false . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

emptiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 equality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 equal (strict equality) . . . . . . . . . . . . . . . . . . . . . . . . . . 100 assert numeric range (inclusive) with within . . . . . . . . . . . . . 100 test numeric value is above given value: . . . . . . . . . . . . . . . . 100 test numeric value is below given value: . . . . . . . . . . . . . . . . 100 mating rec+gular expressions . . . . . . . . . . . . . . . . . . . . 100 test length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 substring inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 assert typeof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 property existence . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 array containment . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 own object keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 responds to, asserting that a given property is a function: . . . . . . 101 Puing it all together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Callba ow 104

e boomerang eect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Parallel execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Appendix - Exercise Results 110

Chapter: Buers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 One Solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Chapter: Event Emier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

CONTENTS

Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Chapter: Low-level File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Exercise 1 - get the size of a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 One solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Exercise 2 - read a unk from a le . . . . . . . . . . . . . . . . . . . . . . . . . . 113 One solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Exercise 3 - read two unks from a le . . . . . . . . . . . . . . . . . . . . . . . . 113 One solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Exercise 4 - Overwrite a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Exercise 5 - append to a le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 One Solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Exercise 6 - ange the content of a le . . . . . . . . . . . . . . . . . . . . . . . . . 116 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Chapter: HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Exercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Chapter: Child processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Chapter: Streaming HTTP Chunked responses . . . . . . . . . . . . . . . . . . . . . . . . 122 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

CONTENTS

xi One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Chapter: UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Chapter: TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Chapter: SSL / TLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Exercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Exercise 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 One solution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Credits
is second revision of this book could not have been done without the the help of Timothy Kevin Oxley, who co-reviewed end edited most of this book.

About this book

is book was built using Node.js, Express, the Jade templating engine, wkhtmltopdf, bash scripts, git and github. Version: 1.1

ii

Acknowledgements
Im grateful to my dear wife Susana for puing up with my mood swings about this book, to my Father for teaing me how to program Basic on a ZX Spectrum when I was 10 years old, to my Parents and my Grandmother Maria do Carmo for oering me my rst IBM PC clone a lile later and to my friend Pedro Mendes who gave me the idea for making this book.

iii

Introduction
At the European JSConf 2009, a young programmer by the name of Ryan Dahl, introduced a project he had been working on. is project was a platform that combining Googles V8 Javascript engine and an event loop. e project then took a dierent direction from other server-side Javascript platforms: all I/O primitives were event-driven, and there was no way around it. Leveraging the power and simplicity of Javascript, it turned the dicult task of writing asynronous applications into an easy one. Since receiving a standing ovation at the end of his talk, Dahls project has been met with unprecedented growth, popularity and adoption. e project was named Node.js, now known to developers simply as Node. Node provides purely evented, non-bloing infrastructure for building highly concurrent soware.

Node allows you to easily construct fast and scalable network services.

Why the sudden, exponential popularity?

Server-side Javascript has been around for some time, what makes this platform so appealing? In previous server-side Javascript implementations, javascript was the raison detre, and the approa focussed on translating common practices from other platforms like Ruby, PERL and Python, into Javascript. Node takes a leap from this and says: Lets use the successful event-driven programming model of the web and use it to make an easy way to build scalable servers. And lets make it the only way people can do anything on this platform.. It can be argued that Javascript itself contributed to mu of Nodes success, but that would not explain why other the server-side projects proceeding Node have not yet come close in popularity. e ubiquity of Javascript surely has played a role, but, as Ryan Dahl points out, unlike other SSJS aempts, a unifying client/server language was not the primary goal for Node. In the perspective of this author, there are three factors contributing to Nodes success: 1. Node is Easy - Node makes event-driven I/O programming, the best way to do I/O programming, mu easier to understand and aieve than ever before. 2. Node is Lean - Node does not try to solve all problems. It lays the foundation and supports the basic internet protocols using clean, functional APIs. 3. Node does not Compromise - Node does not try to be compatible with pre-existing soware, it takes a fresh look at what many believe is the right direction. iv

Introduction

What does this book cover?

We will analyze what makes Node a dierent proposal to other server-side solutions, why you should use it, and how to get started. We will start with an overview but quily dive into some code module-by-module. By the end of this book you should be able to build and test your own Node modules, service producers/consumers and feel comfortable using Nodes conventions and API.

What does this book not cover?

is book does not aempt to cover the complete Node API. Instead, we will cover what the author thinks is required to build most applications he would build on Node. is book does not cover any Node frameworks; Node is a great tool for building frameworks and many are available, su as cluster management, inter-process communication, web frameworks, network trac collection tools, game engines and many others. Before you dive into any of those you should be familiar with Nodes infrastructure and what it provides to these building blos.

Prerequisites
is book does not assume you have any prior knowledge of Node, but the code examples are wrien in Javascript, so familiarity with the Javascript language will help.

Exercises
is book has exercises in some apters. At the end of this book you can nd the exercise solutions, but I advise you to try do them yourself. Consult this book or use the comprehensive API documentation on the ocial hp://nodejs.org website.

Source code
You can nd some of the source code and exercises used in this book on GitHub: hps://github.com/pgte/handson_nodejs_source_code or you can download it directly: hps://github.com/pgte/handson_nodejs_source_code/zipball/master
http://nodejs.org https://github.com/pgte/handson_nodejs_source_code https://github.com/pgte/handson_nodejs_source_code/zipball/master

Introduction

vi

Where will this book lead you?

By the end of it, you should understand the Node API and be able to pursue the exploration of other things built on top of it, being adaptors, frameworks and modules. Lets get started!

Chapter Overview
While this book does not need to be read from cover to cover, it will help since some common concepts presented earlier in the book will probably not be repeated.

Why?
Why does Node use event-driven programming and Javascript together? Why is Javascript a great language?

Starting up
How to install Node and get Node modules using NPM.

Understanding Basics
How Node loads modules, how the Event Loop functions and what to look out for in order not to blo it.

API Quick Tour

i overview of Nodes core modules.

Utilities
Useful Node utilities.

Buers
Learn to create, modify and access buer data, an essential part of the Node fundamentals.

Event Emitter
How the Event Emier paern is used throughout Node and how to use it for exibility in your code.

Timers
Nodes timer API, reminiscent of browsers. vii

Chapter Overview

viii

Low-level File System

How to use Node to open, read and write les.

HTTP
Nodes ri HTTP server and client implementation.

Streams
e riness of this great abstraction in Node.

TCP Server
ily setup a bare TCP server.

UNIX Sockets
How to use UNIX soets and use them to pass le descriptors around.

Datagrams (UDP)
e power of datagrams in Node

Child Processes
Launing, wating, piping and killing other processes.

Streaming HTTP Chunked Responses

HTTP in Node is streamable from the get go.

TLS / SSL
How to provide and consume secure streams.

HTTPS
How to build a secure HTTPS server or client.

Chapter Overview

ix

Making Modules
How to make your app more modular.

Debugging
How to debug your Node app.

Automated Unit Testing

How to unit test your modules.

Callback Flow
How to manage intricate callba ow in a sane way.

Why?
Why the event loop?
e Event Loop is a soware paern that facilitates non-bloing I/O (network, le or inter-process communication). Traditional bloing programming does I/O in the same fashion as regular function calls; processing may not continue until the operation is complete. Here is some pseudo-code that demonstrates bloing I/O:
1 2 3 4 5

var post = db.query('SELECT * FROM posts where id = 1'); // processing from this line onward cannot execute // until the line above completes doSomethingWithPost(post); doSomethingElse();

What is happening here? While the database query is being executed, the whole process/thread idles, waiting for the response. is is called bloing. e response to this query may take many thousands of CPU cycles, rendering the entire process unusable during this time. e process could have been servicing other client requests instead of just waiting. is does not allow you to parallelize I/O su as performing another database query or communicating with a remote web service, without involving concurrent programming triery. e call sta becomes frozen, waiting for the database server to reply. is leaves you with two possible solutions to keep the process busy while its waiting: create more call stas or use event callbas.

Solution 1: Create more call stacks

In order for your process to handle more concurrent I/O, you have to have more concurrent call stas. For this, you can use threads or some kind of cooperative multi-threading seme like coroutines, bers, continuations, etc. e multi-threaded concurrency model can be very dicult to congure, understand and debug, mainly because of the complexity of synronization when accessing shared information; you never know when the thread you are running is going to be preempted, whi can lead to strange and inconsistent bugs creeping up when the threads interleaving is not as expected. On the other hand, cooperative multi-threading is a tri where you have more than one sta, and ea thread of execution explicitly de-sedules itself to give time to another thread. is can relax the synronization requirements but can become complex and error-prone, since the thread seduling is le at the hands of the programmers. 1

Why?

Solution 2: Use event callbacks

An event callba is a function that gets invoked when something signicant happens (e.g. the result of a database query is available.) To use event callbas on the previous example, you could ange it like so:
1 2 3 4 5 6 7 8

callback = function(post) { // this will only execute when the db.query function returns. doSomethingWithPost(post); }; db.query('SELECT * FROM posts where id = 1', callback); // this will execute independent of the returned status // of the db.query call. doSomethingElse();

Here you are dening a function to be invoked when the db operation is complete, then passing this function as an callba argument to the db query operation. e db operation becomes responsible for executing the callba when it completes. You can use an inline anonymous function to express this in a more compact fashion:
1 2 3 4 5 6 7 8 9

db.query('SELECT * FROM posts where id = 1', function(post) { // this will only execute when the db.query function returns. doSomethingWithPost(post); } ); // this will execute independent of the returned status // of the db.query call. doSomethingElse();

While db.query() is executing, the process is free to continue running doSomethingElse(), and even service new client requests.

For quite some time, the C systems-programming haer community has known that event-driven programming is the best way to scale a server to handle many concurrent connections. It has been known to be more ecient regarding memory: less context to store, and time: less context-switing. is knowledge has been inltrating other platforms and communities: some of the most wellknown event loop implementations are Rubys Event Maine, Perls AnyEvent and Pythons Twisted, and there plenty of others.

Why?

3 Tip: For more info about event-driven server implementations, see hp://en.wikipedia.org/wiki/Reactor_paern.

Implementing an application using one of these frameworks requires framework-specic knowledge and framework-specic libraries. For example: when using Event Maine, you should avoid using synronous libraries (i.e. most libraries). To gain the benet of not bloing, you are limited to using only asynronous libraries, built specically for Event Maine. Your server will not be able to scale optimally if the event loop is constantly bloing whi prevents timely processing of I/O events. If you oose to go down the non-bloing rabbit hole, you have to go all the way down, never compromising, and hope you come out on the other side in one piece. Node has been devised as a non-bloing I/O server platform from day one, so generally you should expect everything built on top of it is non-bloing. Since Javascript itself is very minimal and does not impose any way of doing I/O (it does not have a standard I/O library), Node has a clean slate to build upon.

Why Javascript?
Ryan Dahl began this project building a C platform, but maintaining the context between callbas was too complicated and could to poorly structured code, so he then turned to Lua. Lua already has several bloing I/O libraries and this mix of bloing and non-bloing could confuse average developers and prevent many of them from building scalable applications, so Lua wasnt ideal either. Dahl then thought to Javascript. Javascript has closures and rst-class functions, making it indeed a powerful mat with evented I/O programming. Closures are functions that inherit the variables from their enclosing environment. When a function callba executes it will magically remember the context in whi it was declared, along with all the variables available in that context and any parent contexts. is powerful feature is at the heart of Nodes success among programming communities. In the web browser, if you want to listen for an event, a buon cli for instance, you may do something like:
1 2 3 4 5

var clickCount = 0; document.getElementById('mybutton').onclick = function() { clickCount ++;

http://en.wikipedia.org/wiki/Reactor_pattern

Why?

4
alert('Clicked ' + clickCount + ' times.');

6 7 8

};

or, using jery:

1 2 3 4 5

var clickCount = 0; $('button#mybutton').click(function() { clickedCount ++; alert('Clicked ' + clickCount + ' times.'); });

In both examples we assign or pass a function as an argument, whi may be executed later. e cli handling function has access every variable in scope at the point where the function is declared, i.e. practically speaking, the cli handler has access to the cliCount variable, declared in the parent closure. Here we are using a global variable, cliCount, where we store the number of times the user has clied a buon. We can also avoid having a global variable accessible to the rest of the system, by wrapping it inside another closure, making the clied variable only accessible within the closure we created:
1 2 3 4 5 6 7

(function() { var clickCount = 0; $('button#mybutton').click(function() { clickCount ++; alert('Clicked ' + clickCount + ' times.'); }); })();

In line 7 we are invoking a function immediately aer dening it. If this is strange to you, dont worry! We will cover this paern later.

Why?

How I Learned to Stop Fearing and Love Javascript

Javascript has good and bad parts. It was created in 1995 by Netscapes Brendan Ei, in a rush to ship the latest version of the Netscape web browser. Due to this rush some good, even wonderful, parts got into Javascript, but also some bad parts.

is book will not cover the distinction between Javascript good and bad parts. (For all we know, we will only provide examples using the good parts.) For more on this topic you should read Douglas Croford book named Javascript, e Good Parts, edited by OReilly.

In spite of its drawbas, Javascript quily - and somewhat unpredictably - became the de-facto language for web browsers. Ba then, Javascript was used to primarily to inspect and manipulate HTML documents, allowing the creation the rst dynamic, client-side web applications. In late 1998, the World Wide Web Consortium (W3C), standardized the Document Object Model (DOM), an API devised to inspect and manipulate HTML documents on the client side. In response to Javascripts quirks and the initial hatred towards the DOM API, Javascript quily gained a bad reputation, also due to some incompatibilities between browser vendors (and sometimes even between products from the same vendor!). Despite mild to full-blown hate in some developer communities, Javascript became widely adopted. For beer of for worse, today Javascript is the most widely deployed programming language on Earth. If you learn the good features of the language - su as prototypical inheritance, function closures, etc. - and learn to avoid or circumvent the bad parts, Javascript can be a very pleasant language to work in.

Function Declaration Styles

A function can be declared in many ways in Javascript. e simplest is declaring it anonymously:
1 2 3

function() { console.log('hello'); }

Here we declare a function, but its not of mu use, because do not invoke it. Whats more, we have no way to invoke it as it has no name. We can invoke an anonymous function in-place:

Why?

1 2 3

(function() { console.log('hello'); })();

Here we are executing the function immediately aer declaring it. Notice we wrap the entire function declaration in parenthesis. We can also name functions like this:
1 2 3

function myFunction () { console.log('hello'); }

Here we are declaring a named function with the name: myFunction. myFunction will be available inside the scope its declared
1

myFunction();

and also within inner scopes:

1 2 3 4 5 6 7

function myFunction () { console.log('hello'); } (function() { myFunction(); })();

A result of Javascript treating functions as rst-class objects means we can assign a function to a variable:
1 2 3

var myFunc = function() { console.log('hello'); }

is function is now available as the value of the myFunc variable. We can assign that function to another variable:
1

var myFunc2 = myFunc;

And invoke them just like any other function:

Why?

1 2

myFunc(); myFunc2();

We can mix both teniques, having a named function stored in a variable:

1 2 3 4

var myFunc2 = function myFunc() { console.log('hello'); } myFunc2();

Note though, we cannot access myFunc from outside the scope of myFunc itself! We can then use a variable or a function name to pass variables into functions like this:
1 2 3 4 5

var myFunc = function() { console.log('hello'); } console.log(myFunc);

or simply declare it inline if we dont need it for anything else:

1 2 3

console.log(function() { console.log('hello'); });

Functions are rst-class objects

In fact, there are no second-class objects in Javascript. Javascript is the ultimate object-oriented language, where everything is indeed, an object. As that, a function is an object where you can set properties, pass it around inside arguments and return them. Always a pleasure. Example:
1 2 3 4 5 6 7 8

var schedule = function(timeout, callbackfunction) { return { start: function() { setTimeout(callbackfunction, timeout) } }; };

Why?

9 10 11 12 13 14 15 16 17 18 19

(function() { var timeout = 1000; // 1 second var count = 0; schedule(timeout, function doStuff() { console.log(++ count); schedule(timeout, doStuff); }).start(timeout); })(); // "timeout" and "count" variables // do not exist on this scope.

In this lile example we create a function and store it in a function called sedule (starting on line 1). is function just returns an object that has one property called start (line 3). is value of the start property is a function that, when called, sets a timeout (line 4) to call a function that is passed in as the timeout argument. is timeout will sedule callba function to be called within the number of seconds dened in the timeout variable passes. On line 9 we declare a function that will immediately be executed on line 16. is is a normal way to create new scopes in Javascript. Inside this scope we create 2 variables: timeout (line 10) and count (line 11). Note that these variables will not be accessible to the outer scope. en, on line 12, we invoke the sedule function, passing in the timeout value as rst argument and a function called doStu as second argument. When the timeout occurs, this function will increment the variable count and log it, and also call the sedule all over again. So in this small example we have: functions passed as argument, functions to create scope, functions to serve as asynronous callbas and returning functions. We also here present the notions of encapsulation (by hiding local variables form the outside scope) and recursion (the function is calling itself at the end). In Javascript you can even set and access aributes in a function, something like this:
1 2 3 4 5 6

var myFunction = function() { // do something crazy }; myFunction.someProperty = 'abc'; console.log(myFunction.someProperty); // #=> "abc"

Javascript is indeed a powerful language, and if you dont already do, you should learn it and embrace its good parts.

Why?

JSLint
Its not to be covered here, but Javascript indeed has some bad parts, and they should be avoided at all costs. One tool thats proven invaluable to the author is JSLint, by Douglas Croford. JSLint analyzes your Javascript le and outputs a series of errors and warnings, including some known misuses of Javascript, like using globally-scoped variables (like when you forget the var keyword), and freezing values inside iteration that have callbas that use them, and many others that are useful. JSLint can be installed using
1

$ npm install jslint

If you dont have NPM installed see section about NPM.

and can be run from the command line like this:

1

$ jslint myfile.js

To the author of this book, JSlint has proven itself to be an invaluable tool to guarantee he doesnt fall into some common Javascript traps.

Javascript versions Javascript is a standard with its own name - ECMAScript - and it has gone through various iterations. Currently Node natively supports everything the V8 Javascript engine supports ECMA 3rd edition and parts of the new ECMA 5th edition. ese parts of ECMA 5 are nicely documented on the following github wiki page: hps://github.com/joyent/node/wiki/ECMA-5-Mozilla-Features-Implemented-in-V8

index.html#npm https://github.com/joyent/node/wiki/ECMA-5-Mozilla-Features-Implemented-in-V8

Why?

10

Handling callbacks
In Node you can implement your own functions that perform asynronous I/O. To do so, you can accept a callba function. You will invoke this function when the I/O is done.
1 2 3 4 5 6 7

var myAsyncFunction = function(someArgument1, someArgument2, callback) { // simulate some I/O was done setTimeout(function() { // 1 second later, we are done with the I/O, call the callback callback(); }, 1000) }

On line 3 we are invoking a setTimeout to simulate the delay and asynronism of an I/O call. is setTimeout function will call the rst argument - a function we declare inline - aer 1000 milliseconds (the second argument) have gone by. is inline function will then call the callba that was passed as the third argument to myAsyncFunction, notifying the caller the operation has ended. Using this convention and others (like the Event Emier paern whi we will cover later) you can embrace Javascript as the ultimate language for event-driven applications.

Tip: To follow the Node convention, this function should receive the error (or null if there was no error) as rst argument, and then some real arguments if you wish to do so.
1 2 3 4

fs.open('/path/to/file', function(err, fd) { if (err) { /* handle error */; return; } console.log('opened file and got file descriptor ' + fd); })

Here we are using a Node API function fs.open that receives 2 arguments: the path to the le and a function, whi will be invoked with an error or null on the rst argument and a le descriptor on the second.

Why?

11

References
Event Maine: hp://rubyeventmaine.com/ Twisted: hp://twistedmatrix.com/trac/ AnyEvent: hp://soware.smorp.de/pkg/AnyEvent.html Javascript, the Good Parts - Douglas Croford - OReilly hp://www.amazon.com/exec/obidos/ASIN/0596517742/wrrrldwideweb JSLint hp://www.jslint.com/

http://rubyeventmachine.com/ http://twistedmatrix.com/trac/ http://software.schmorp.de/pkg/AnyEvent.html http://www.amazon.com/exec/obidos/ASIN/0596517742/wrrrldwideweb http://www.jslint.com/

Starting up
Install Node
e typical way of installing Node on your development maine is by following the steps on the nodejs.org website. Node should install out of the box on Linux, Macintosh, and Solaris.

With some eort you should be able to get it running on other Unix platforms and Windows (either via Cygwin or MinGW).

Node has several dependencies, but fortunately most of them are distributed along with it. If you are building from source you should only need 2 things: python - version 2.4 or higher. e build tools distributed with Node run on python. libssl-dev - If you plan to use SSL/TLS encryption in your networking, youll need this. Libssl is the library used in the openssl tool. On Linux and Unix systems it can usually be installed with your favorite paage manager. e lib comes preinstalled on OS X.

Pi the latest stable version and download it:

1

$ wget http://nodejs.org/dist/node-v0.4.7.tar.gz

Expand it:
1

$ tar xvfz node-v0.4.7.tar.gz

Build it:
http://nodejs.org

12

Starting up

13

1 2 3 4

$ $ $ $

cd node-v0.4.7 ./configure make make install

Tip: if you are having permission problems on this last step you should run the install step as a super user like by:
1

$ sudo make install

Aer you are done, you should be able to run the node executable on the command line:
1 2

$ node -v v0.4.7

e node executable can be executed in two main fashions: CLI (command-line interface) or le. To laun the CLI, just type
1

$ node

and you will get a Javascript command line prompt, whi you can use to evaluate Javascript. Its great for kiing the tires and trying out some stu quily. You can also laun Node on a le, whi will make Node parse and evaluate the Javascript on that le, and when it ends doing that, it enters the event loop. Once inside the event loop, node will exit if it has nothing to do, or will wait and listen for events. You can laun Node on a le like this:
1

$ node myfile.js

or, if you wish, you can also make your le directly executable by anging the permissions like this:
1

$ chmod o+x myfile.js

and insert the following as the rst line of the le:

1

#!/usr/bin/env node

You can then execute the le direclty:

1

$ ./myfile.js

Starting up

14

NPM - Node Package Manager

NPM has become the standard for managing Node paages throughout time, and tight collaboration between Isaac Slueter - the original author of NPM - and Ryan Dahl - the author and maintainer on Node - has further tightened this relationship to the point where, starting at version 0.4.0, Node supports the paage.json le format format to indicate dependencies and paage starting le. To install it you type:
1

$ curl http://npmjs.org/install.sh sh

If that fails, try this on a temporary directory:

1 2 3

$ git clone http://github.com/isaacs/npm.git $ cd npm $ sudo make install

NPM commands
NPM can be used on the command line. e basic commands are:

npm ls [filter]
Use this to see the list of all paages and their versions (npm ls with no lter), or lter by a tag (npm lter tag). Examples: List all installed paages:
1

$ npm ls installed

List all stable paages:

1

$ npm ls stable

You can also combine lters:

Starting up

15

$ npm ls installed stable

You can also use npm ls to sear by name:

1

$ npm ls fug

(this will return all paages that have fug inside its name or tags) You can also query it by version, prexed with the @ aracter:
1

$ npm ls @1.0

npm install package[@filters]

With this command you can install a paage and all the paages it depends on. To install the latest version of a paage do:
1

$ npm install package_name

Example:
1

$ npm install express

To install a specic version of a paage do:

1

$npm install package_name@version

Example:
1

$ npm install [email protected]

To install the latest within a version range you can specify, for instance:
1

$ npm install express@">=0.1.0

You can also combine many lters to select a specic version, combining version range and / or tags like this:
1

npm install sax@">=0.1.0 <0.2.0" bench supervisor

Starting up

16

npm rm package_name[@version] [package_name[@version] ...]

Use this command to uninstall paages. If versions are omied, then all the found versions are removed. Example:
1

$ npm rm sax

npm view [@] [[.]...]

To view all of a paage info. Defaults to the latest version if version is omied. View the latest info on the connect paage:
1

$ npm view connect

View information about a specic version:

1

$ npm view [email protected]

Further on we will look more into NPM and how it can help us bundle and freeze application dependencies.

Understanding
Understanding the Node event loop
Node makes evented I/O programming simple and accessible, puing speed and scalability on the ngertips of the common programmer. But the event loop comes with a price. Even though you are not aware of it (and Node makes a good job at this), you should understand how it works. Every good programmer should know the intricacies of the platforms he / she is building for, its dos and donts, and in Node it should be no dierent.

An event-queue processing loop

You should think of the event loop as a loop that processes an event queue. Interesting events happen, and when they do, they go in a queue, waiting for their turn. en, there is an event loop popping out these events, one by one, and invoking the associated callba functions, one at a time. e event loop pops one event out of the queue and invokes the associated callba. When the callba returns the event loop pops the next event and invokes the associated callba function. When the event queue is empty, the event loop waits for new events if there are some pending calls or servers listening, or just quits of there are none. So, lets jump into our rst Node example. Write a le named hello.js with the following content: Source code in: apters/understanding/1_hello.js
1 2 3 4

setTimeout(function() { console.log('World!'); }, 2000); console.log('Hello');

Run it using the node command line tool:

1

$ node hello.js

You should see the word Hello wrien out, and then, 2 seconds later, World!. Shouldnt World! have been wrien rst since it appears rst on the code? No, and to answer that properly we must analyze what happens when executing this small program. On line 1 we declare an anonymous function that prints out World!. is function, whi is not yet executed, is passed in as the rst argument to a setTimeout call, whi sedules this function to run in 2000 milliseconds. en, on line 4, we output Hello into the console. 17

Understanding

18

Two seconds later the anonymous function we passed in as an argument to the setTimeout call is invoked, printing World!. So, the rst argument to the setTimeout call is a function we call a callba. Its a function whi will be called later, when the event we set out to listen to (in this case, a time-out of 2 seconds) occurs.

We can also pass callba functions to be called on events like when a new TCP connection is established, some le data is read or some other type of I/O event.

Aer our callba is invoked, printing World, Node understands that there is nothing more to do and exits.

Callbacks that will generate events

Lets complicate this a bit further. Lets keep Node busy and keep on seduling callbas like this: Source code in apters/understanding/2_repeat.js
1 2 3 4 5 6

(function schedule() { setTimeout(function() { console.log('Hello World!'); schedule(); }, 1000); })();

Here we are wrapping the whole thing inside a function named sedule, and we are invoking it immediately aer declaring it on line 6. is function will sedule a callba to execute in 1 second. is callba, when invoked, will print Hello World! and then run sedule again. On every callba we are registering a new one to be invoked one second later, never leing Node nish. is lile script will just keep printing Hello World.

Dont block!
Node primary concern and the main use case for an event loop is to create highly scalable servers. Since an event loop runs in a single thread, it only processes the next event when the callba nishes. If you could see the call sta of a busy Node application you would see it going up and

Understanding

19

down really fast, invoking callbas and piing up the next event in line. But for this to work well you have to clear the event loop as fast as you can. ere are two main categories of things that can blo the event loop: synronous I/O and big loops. Node API is not all asynronous. Some parts of it are synronous like, for instance, some le operations. Dont worry, they are very well marked: they always terminate in Sync - like fs.readFileSync - , and they should not be used, or used only when initializing. On a working server you should never use a bloing I/O function inside a callba, since youre bloing the event loop and preventing other callbas - probably belonging to other client connections - from being served.

One function that is synronous and does not end in Sync is the require function, whi should only be used when initializing an app or a module.

Tip: Dont put a require statement inside a callba, since it is synronous and thus will slow down your event loop.

e second category of bloing scenarios is when you are performing loops that take a lot of time, like iterating over thousands of objects or doing complex time-taking operations in memory. ere are several teniques that can be used to work around that, whi well cover later. Here is a case where we present some simple code that blos the event loop:
1 2 3 4 5 6 7 8 9 10 11

var open = false; setTimeout(function() { open = true; }, 1000) while(!open) { // wait } console.log('opened!');

Understanding

20

Here we are seing a timeout, on line 3, that invokes a function that will set the open variable to true. is function is set to be triggered in one second. On line 7 we are waiting for the variable to become true. We could be lead to believe that, in one second the timeout will happen and set open to true, and that the while loop will stop and that we will get opened! (line 11) printed. But this never happens. Node will never execute the timeout callba because the event loop is stu on this while loop started on line 7, never giving it a ance to process the timeout event!

Understanding Modules
Client-side Javascript has a bad reputation also because of the common namespace shared by all scripts, whi can lead to conicts and security leaks. Node implements the CommonJS modules standard, where ea module is separated from the other modules, having a separate namespace to play with, and exporting only the desired properties. To include an existing module you can use the require function like this:
1

var module = require('module_name');

is will fet a module that was installed by npm. If you want to author modules (as you should when doing an application), you can also use the relative notation like this:
1

var module = require("./path/to/module_name");

is will fet the module relatively to the current le we are executing. We will cover creating modules on a later section.

In this format you can use an absolute path (starting with /) or a relative one (starting with .);

Modules are loaded only once per process, that is, when you have several require calls to the same module, Node caes the require call if it resolves to the same le. Whi leads us to the next apter.

Understanding

21

How Node resolves a module path

So, how does node resolve a call to require(module_path)? Here is the recipe:

Core modules
ere are a list of core modules, whi Node includes in the distribution binary. If you require one of those modules, Node just returns that module and the require() ends.

Modules with complete or relative path

If the module path begins with ./ or /, Node tries to load the module as a le. If it does not succeed, it tries to load the module as a directory.

As a le
When loading as a le, if the le exists, Node just loads it as Javascript text. If not, it tries doing the same by appending .js to the given path. If not, it tries appending .node and load it as a binary add-on.

As a directory
If appending /paage.json is a le, try loading the paage denition and look for a main eld. Try to load it as a le. If unsuccessful, try to load it by appending /index to it.

As an installed module
If the module path does not begin with . or / or if loading it with complete or relative paths does not work, Node tries to load the module as a module that was previously installed. For that it adds /node_modulesto the current directory and tries to load the module from there. If it does not succeed it tries adding /node_modules to the parent directory and load the module from there. If it does not succeed it moves again to the parent directory and so on, until either the module is found or the root of the tree is found. is means that you can put your bundle your Node modules into your app directory, and Node will nd those. Later we will see how using this feature together with NPM we can bundle and freeze your application dependencies.

Understanding

22

Also you can, for instance, have a node_modules directory on the home folder of ea user, and so on. Node tries to load modules from these directories, starting rst with the one that is closest up the path.

API quick tour

Node provides a platform API that covers mainly 4 aspects: processes lesystem networking utilities

is book is not meant to be a comprehensive coverage of the whole Node API. For that you should consult the Node online documentation on hp://nodejs.org.

Processes
Node allows you to analyze your process (environment variables, etc.) and manage external processes. e involved modules are:

process
Inquire the current process to know the PID, environment variables, platform, memory usage, etc.

child_process
Spawn and kill new processes, execute commands and pipe their outputs.

File system
Node also provides a low-level API to manipulate les, whi is inspired by the POSIX standard, and is comprised by the following modules:
http://nodejs.org.

23

API qui tour

24

fs
File manipulation: create, remove, load, write and read les. Create read and write streams (covered later).

path
Normalize and join le paths. Che if a le exists or is a directory.

Networking
net
Create a TCP server or client.

dgram
Receive and send UDP paets.

http
Create an HTTP server or Client.

tls (ssl)
e tls module uses OpenSSL to provide Transport Layer Security and/or Secure Soet Layer: encrypted stream communication.

https
Implementing hp over TLS/SSL.

dns
Asynronous DNS resolution.

Utilities
console
Node provides a global console object to whi you can output strings using:
1

console.log("Hello");

is simply outputs the string into the process stdout aer formaing it. You can pass in, instead of a string, an object like this:
1 2

var a = {1: true, 2: false}; console.log(a); // => { '1': true, '2': false }

In this case console.log outputs the object using util.inspect (covered later); You can also use string interpolation like this:
1 2 3 4

var a = {1: true, 2: false}; console.log('This is a number: %d, and this is a string: %s,' + 'and this is an object outputted as JSON: %j', 42, 'Hello', a);

Whi outputs:
1 2

This is a number: 42, and this is a string: Hello, and this is an object o\ utputted as JSON: {"1":true,"2":false}

console also allows you to write into the the stderr using:
1

console.warn("Warning!");

and to print a sta trace:

1 2 3 4 5

console.trace(); Trace: at [object Context]:1:9 at Interface. (repl.js:171:22)

25

Utilities

26

6 7 8 9 10 11 12 13

at Interface.emit (events.js:64:17) at Interface._onLine (readline.js:153:10) at Interface._line (readline.js:408:8) at Interface._ttyWrite (readline.js:585:14) at ReadStream. (readline.js:73:12) at ReadStream.emit (events.js:81:20) at ReadStream._emitKey (tty_posix.js:307:10) at ReadStream.onData (tty_posix.js:70:12)

util
Node has an util module whi whi bundles some functions like:
1 2

var util = require('util'); util.log('Hello');

whi outputs a the current timestamp and the given string like this:
1

Mar 16:38:31 - Hello

e inspect function is a nice utility whi can aid in qui debugging by inspecting and printing an object properties like this:
1 2 3 4

var util = require('util'); var a = {1: true, 2: false}; console.log(util.inspect(a)); // => { '1': true, '2': false }

util.inspect accepts more arguments, whi are:

1

util.inspect(object, showHidden, depth = 2, showColors);

the second argument, showHidden should be turned on if you wi inspect to show you nonenumerable properties, whi are properties that belong to the object prototype ain, not the object itself. depth, the third argument, is the default depth on the object graph it should show. is is useful for inspecting large objects. To recurse indenitely, pass a null value.

Tip: util.inspect keeps tra of the visited objects, so circular dependencies are no problem, and will appear as [Circular] on the outpued string.

Utilities

27

e util module has some other niceties, su as inheritance setup and stream pumping, but they belong on more appropriate apters.

Buers
Natively, Javascript is not very good at handling binary data. So Node adds a native buer implementation with a Javascript way of manipulating it. Its the standard way in Node to transport data.

Generally, you can pass buers on every Node API requiring data to be sent. Also, when receiving data on a callba, you get a buer (except when you specify a stream encoding, in whi case you get a String). is wil be covered later.

You can create a Buer from an UTF8 string like this:

1

var buf = new Buffer('Hello World!');

You can also create a buer from strings with other encodings, as long as you pass it as the second argument:
1

var buf = new Buffer('8b76fde713ce', 'base64');

Accepted encodings are: ascii, utf8 and base64. or you can create a new empty buer with a specic size:
1

var buf = new Buffer(1024);

and you can manipulate it:

1

buf[20] = 56; // set byte 20 to 56

You can also convert it to a UTF-8-encoded string:

1

var str = buf.toString();

or into a string with an alternative encoding: 28

Buers

29

var str = buf.toString('base64');

UTF-8 is the default encoding for Node, so, in a general way, if you omit it as we did on the buer.toString() call, UTF-8 will be assumed.

Slice a buer
A buer can be sliced into a smaller buer by using the appropriately named slice() method like this:
1 2

var buffer = new Buffer('this is the string in my buffer'); var slice = buffer.slice(10, 20);

Here we are slicing the original buer that has 31 bytes into a new buer that has 10 bytes equal to the 10th to 20th bytes on the original buer. Note that the slice function does not create new buer memory: it uses the original untoued buer underneath.

Tip: If you are afraid you will be wasting precious memory by keeping the old buer around when slicing it, you can copy it into another like this:

Copy a buer
You can copy a part of a buer into another pre-allocated buer like this:
1 2 3 4 5 6

var buffer = new Buffer('this is the string in my buffer'); var slice = new Buffer(10); var targetStart = 0, sourceStart = 10, sourceEnd = 20; buffer.copy(slice, targetStart, sourceStart, sourceEnd);

Here we are copying part of buer into slice, but only positions 10 through 20.

Buers

30

Buer Exercises
Exercise 1
Create an uninitialized buer with 100 bytes length and ll it with bytes with values starting from 0 to 99. And then print its contents.

Exercise 2
Do what is asked on the previous exercise and then slice the buer with bytes ranging 40 to 60. And then print it.

Exercise 3
Do what is asked on exercise 1 and then copy bytes ranging 40 to 60 into a new buer. And then print it.

Event Emitter
On Node many objects can emit events. For instance, a TCP server can emit a connect event every time a client connects. Or a le stream request can emit a data event.

.addListener
You can listen for these events by calling one of these objects addListener method, passing in a callba function. For instance, a le ReadStream can emit a data event every time there is some data available to read. Instead of using the addListener function, you can also use on, whi is exactly the same thing:
1 2 3 4 5 6 7 8

var fs = require('fs'); // get the fs module var readStream = fs.createReadStream('/etc/passwd'); readStream.on('data', function(data) { console.log(data); }); readStream.on('end', function() { console.log('file ended'); });

Here we are binding to the readStreams data and end events, passing in callba functions to handle ea of these cases. When one of these events happens, the readStream will call the callba function we pass in. You can either pass in an anonymous function as we are doing here, or you can pass a function name for a function available on the current scope, or even a variable containing a function.

.once
You may also want to listen for an event exactly once. For instance, if you want to listen to the rst connection on a server, you should do something like this:
1 2 3

server.once('connection', function (stream) { console.log('Ah, we have our first user!'); });

is works exactly like our on example, except that our callba function will be called at most once. It has the same eect as the following code: 31

Event Emier

32

1 2 3 4 5

function connListener(stream) { console.log('Ah, we have our first user!'); server.removeListener('connection', connListener); } server.on('connection', connListener);

Here we are using the removeListener, whi also belongs to the EventEmier paern. It accepts the event name and the function it should remove.

.removeAllListeners
If you ever need to, you can also remove all listeners for an event from an Event Emier by simply calling
1

server.removeAllListeners('connection');

Creating an Event Emitter

If you are interested on using this Event Emier paern - and you should - throughout your application, you can. You can create a pseudo-class and make it inherit from the EventEmier like this:
1 2 3 4 5 6 7 8 9 10

var EventEmitter = require('events').EventEmitter, util = require('util'); // Here is the MyClass constructor: var MyClass = function(option1, option2) { this.option1 = option1; this.option2 = option2; } util.inherits(MyClass, EventEmitter);

util.inherits is seing up the prototype ain so that you get the EventEmier prototype methods available to your MyClass instances.

is way instances of MyClass can emit events:

Event Emier

33

1 2 3

MyClass.prototype.someMethod = function() { this.emit('custom event', 'some arguments'); }

Here we are emiting an event named custom event, sending also some data (some arguments in this case); Now clients of MyClass instances can listen to custom events events like this:
1 2 3 4

var myInstance = new MyClass(1, 2); myInstance.on('custom event', function() { console.log('got a custom event!'); });

Tip: e Event Emier is a nice way of enforcing the decoupling of interfaces, a soware design tenique that improves the independence from specic interfaces, making your code more exible.

Event Emitter Exercises

Exercise 1
Build a pseudo-class named Tier that emits a ti event every 1 second.

Exercise 2
Build a script that instantiates one Tier and bind to the ti event, printing TICK every time it gets one.

Timers
Node implements the timers API also found in web browsers. e original API is a bit quirky, but it hasnt been anged for the sake of consistency.

setTimeout
setTimeout lets you sedule an arbitrary function to be executed in the future. An example:
1 2 3 4

var timeout = 2000; // 2 seconds setTimeout(function() { console.log('timed out!'); }, timeout);

is code will register a function to be called when the timeout expires. Again, as in any place in Javascript, you can pass in an inline function, the name of a function or a variable whi value is a function.

You can use setTimeout with a timeout value of 0 so that the function you pass gets executed some time aer the sta clears, but with no waiting. is can be used to, for instance sedule a function that does not need to be executed immediately. is was a tri sometimes used on browser Javascript, but, as we will see, Node process.nextTi() can be used instead of this, and its more ecient.

clearTimeout
setTimeout returns a timeout handle that you can use to disable it like this:
1 2 3

var timeoutHandle = setTimeout(function() { console.log('yehaa!'); }, 1000); clearTimeout(timeoutHandle);

Here the timeout will never execute because we clear it right aer we set it. Another example: 34

Timers

35 Source code in: apters/timers/timers_1.js

1 2 3 4 5 6 7 8

var timeoutA = setTimeout(function() { console.log('timeout A'); }, 2000); var timeoutB = setTimeout(function() { console.log('timeout B'); clearTimeout(timeoutA); }, 1000);

Here we are starting two timers: one with 1 second (timeoutB) and the other with 2 seconds (timeoutA). But timeoutB (whi res rst) unsedules timeoutA on line 7, so timeoutA is never executes - and the program exits right aer line 7 is executed.

setInterval
Set interval is similar to set timeout, but sedules a given function to run every X seconds like this: Source code in: apters/timers/timers_2.js
1 2 3 4

var period = 1000; // 1 second var interval = setInterval(function() { console.log('tick'); }, period);

is will indenitely keep the console logging ti unless you terminate Node. You can unsedule an interval by calling:

clearInterval
clearInterval unsedules a running interval (previous seduled with setInterval ).
1 2

var interval = setInterval(...); clearInterval(interval);

Here we are using the setInterval return value stored on the interval variable to unsedule it on line 2.

Timers

36

process.nextTick
You can also sedule a callba function to run on the next run of the event loop. You can use it like this:
1 2 3 4

process.nextTick(function() { // this runs on the next event loop console.log('yay!'); });

As we saw, this method is preferred to setTimeout(fn, 0) because it is more ecient.

Escaping the event loop

On ea loop, the event loop executes the queued I/O events sequentially by calling the associated callbas. If, on any of the callbas you take too long, the event loop wont be processing other pending I/O events meanwhile. is can lead to waiting customers or tasks. When executing something that may take too long, you can delay the execution until the next event loop, so waiting events will be processed meanwhile. Its like going to the ba of the line on a waiting line. To escape the current event loop you can use process.nextTi() like this:
1 2 3

process.nextTick(function() { // do something });

You can use this to delay processing that is not necessary to do immediately to the next event loop. For instance, you may need to remove a le, but perhaps you dont need to do it before replying to the client. So, you could do something like this:
1 2 3 4 5 6

stream.on('data', function(data) { stream.end('my response'); process.nextTick(function() { fs.unlink('path/to/file'); }) });

Timers

37

A note on tail recursion Lets say you want to sedule a function that does some I/O - like parsing a log le to execute periodically, and you want to guarantee that no two of those functions are executing at the same time. e best way is not to use a setInterval, since you dont have that guarantee. e interval will re no maer if the function has nished its duty or not. Supposing there is an asynronous function called async that performs some IO and that gets a callba to be invoked when nished, and you want to call it every second:
1 2 3 4 5 6

var interval = 1000; // 1 second setInterval(function() { async(function() { console.log('async is done!'); }); }, interval);

If any two async() calls cant overlap, you are beer o using tail recursion like this:
1 2 3 4 5 6 7 8 9

var interval = 1000; // 1 second (function schedule() { setTimeout(function() { async(function() { console.log('async is done!'); schedule(); }); }, interval) })();

Here we are declaring a function named sedule (line 2) and we are invoking it immediately aer we are declaring it (line 9). is function sedules another function to execute within one second (line 3 to 8). is other function will then call async() (line 4), and only when async is done we sedule a new one by calling sedule() again (line 6), this time inside the sedule function. is way we can be sure that no two calls to async execute simultaneously in this context. e dierence is that we probably wont have async called every second (unless async takes no time to execute), but we will have it called 1 second aer the last one nished.

Low-level le-system
Node has a nice streaming API for dealing with les in an abstract way, as if they were network streams, but sometimes you might need to go down a level and deal with the lesystem itself. First, a nice set of utilities:

fs.stat and fs.fstat

You can query some meta-info on a le (or dir) by using fs.stat like this:
1 2 3 4 5 6 7

var fs = require('fs'); fs.stat('/etc/passwd', function(err, stats) { if (err) {console.log(err.message); return; } console.log(stats); //console.log('this file is ' + stats.size + ' bytes long.'); });

If you print the stats object it will be something like:

1 2 3 4 5 6 7 8 9 10 11 12 13 14

{ dev: 234881026, ino: 24606, mode: 33188, nlink: 1, uid: 0, gid: 0, rdev: 0, size: 3667, blksize: 4096, blocks: 0, atime: Thu, 17 Mar 2011 09:14:12 GMT, mtime: Tue, 23 Jun 2009 06:19:47 GMT, ctime: Fri, 14 Aug 2009 20:48:15 GMT }

stats is a Stats instance, with whi you can call:

1 2

stats.isFile() stats.isDirectory()

38

Low-level le-system

39

3 4 5 6 7

stats.isBlockDevice() stats.isCharacterDevice() stats.isSymbolicLink() stats.isFIFO() stats.isSocket()

If you have a plain le descriptor you can use fs.fstat(leDescriptor, callba) instead. More about le descriptors later.

If you are using the low-level lesystem API in Node, you will get le descriptors as a way to represent les. ese le descriptors are plain integer numbers that represent a le in your Node process, mu like in C POSIX APIs.

Open a le
You can open a le by using fs.open like this:
1 2 3 4

var fs = require('fs'); fs.open('/path/to/file', 'r', function(err, fd) { // got fd });

e rst argument to fs.open is the le path. e second argument is the ags, whi indicate the mode with whi the le is to be open. e ags can be r, r+, w, w+, a, or a+. Here is the semantics of ea ag, taken from the fopen man page: r - Open text le for reading. e stream is positioned at the beginning of the le. r+ - Open for reading and writing. e stream is positioned at the beginning of the le.

Low-level le-system

40

w - Truncate le to zero length or create text le for writing. e stream is positioned at the beginning of the le. w+ - Open for reading and writing. e le is created if it does not exist, otherwise it is truncated. e stream is positioned at the beginning of the le. a - Open for writing. e le is created if it does not exist. e stream is positioned at the end of the le. Subsequent writes to the le will always end up at the then current end of le. a+ - Open for reading and writing. e le is created if it does not exist. e stream is positioned at the end of the le. Subsequent writes to the le will always end up at the then current end of le. On the callba function, you get a second argument (fd), whi is a le descriptor- nothing more than an integer that identies the open le, whi you can use like a handler to read and write from.

Read from a le
Once its open, you can also read from a le like this: Source code in: apters/fs/read.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

var fs = require('fs'); fs.open('/var/log/system.log', 'r', function(err, fd) { if (err) { throw err } var readBuffer = new Buffer(1024), bufferOffset = 0, bufferLength = readBuffer.length, filePosition = 100; fs.read(fd, readBuffer,bufferOffset, bufferLength, filePosition, function(err, readBytes) { if (err) { throw err; } console.log('just read ' + readBytes + ' bytes'); if (readBytes > 0) { console.log(readBuffer.slice(0, readBytes)); } }); });

Here we are opening the le, and when its opened we are asking to read a unk of 1024 bytes from it, starting at position 100 (line 9). e last argument to the fs.read call is a callba function (line 10) whi will be invoked when one of the following 3 happens:

Low-level le-system

41

there is an error, something has been read or nothing could be read. On the rst argument, this callba gets an error if there was an one, or null. On the second argument (readBytes) it gets the number of bytes read into the buer. If the read bytes is zero, the le has reaed the end.

Write into a le
To write into a le descriptor you can use fs.write like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

var fs = require('fs'); fs.open('/var/log/system.log', 'a', function(err, fd) { var writeBuffer = new Buffer('writing this string'), bufferOffset = 0, bufferLength = writeBuffer.length, filePosition = null; fs.write( fd, writeBuffer, bufferOffset, bufferLength, filePosition, function(err, written) { if (err) { throw err; } console.log('wrote ' + written + ' bytes'); } ); });

Here we are opening the le in append-mode (a) on line 3, and then we are writing into it (line 8), passing in a buer with the data we want wrien, an oset inside the buer where we want to start writing from, the length of what we want to write, the le position and a callba. In this case we are passing in a le position of null, whi is to say that he writes at the current le position. Here we are also opening in append-mode, so the le cursor is positioned at the end of the le.

Low-level le-system

42

Close Your les On all these examples we did not close the les. is is because these are small simple examples destined to be run and returned. All open les will be closed once the process exists. In real applications you should keep tra of those le descriptors and eventually close them using fs.close(fd[, callba]) when no longer needed.

Advanced Tip: careful when appending concurrently If you are using these low-level le-system functions to append into a le, and concurrent writes will be happening, opening it in append-mode will not be enough to ensure there will be no overlap. Instead, you should keep tra of the last wrien position before you write, doing something like this:
1 2 3 4 5 6 7 8 9 10 11 12

// Appender var fs = require('fs'); var startAppender = function(fd, startPos) { var pos = startPos; return { append: function(buffer, callback) { var oldPos = pos; pos += buffer.length; fs.write(fd, buffer, 0, buffer.length, oldPos, callback); } }; };

Here we declare a function stored on a variable named startAppender. is function starts the appender state (position and le descriptor) and then returns an object with an append function. Now lets do a script that uses this Appender:
1 2 3 4 5 6

// start appender fs.open('/tmp/test.txt', 'w', function(err, fd) { if (err) {throw err; } var appender = startAppender(fd, 0); appender.append(new Buffer('append this!'), function(err) { console.log('appended');

Low-level le-system

43

7 8

}); });

And here we are using the appender to safely append into a le. is function can then be invoked to append, and this appender will keep tra of the last position (line 4 on the Appender), and increments it according to the buer length that was passed in.

Actually, there is a problem with this code: fs.write() may not write all the data we asked it to, so we need to do something a lile bit smarter here: Source code in apters/fs/appender.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

// Appender var fs = require('fs'); var startAppender = function(fd, startPos) { var pos = startPos; return { append: function(buffer, callback) { var written = 0; var oldPos = pos; pos += buffer; (function tryWriting() { if (written < buffer.length) { fs.write(fd, buffer, written, buffer.length - written, oldPos + written, function(err, bytesWritten) { if (err) { callback(err); return; } written += bytesWritten; tryWriting(); } ); } else { // we have finished callback(null); } })(); } } };

Low-level le-system

44

Here we use a function named tryWriing that will try to write, call fs.write, calculate how many bytes have already been wrien and call itself if needed. When it detects it has nished (wrien == buer.length) it calls callba to notify the caller, ending the loop. Dont be frustrated if you dont grasp this on the rst time around. Give it some time to sink in and come ba here aer you have nished reading this book. Also, the appending client is opening the le with mode w, whi truncates the le, and its telling appender to start appending on position 0. is will overwrite the le if it has content. So, a wizer version of the appender client would be:
1 2 3 4 5 6 7 8 9 10 11 12

// start appender fs.open('/tmp/test.txt', 'a', function(err, fd) { if (err) {throw err; } fs.fstat(fd, function(err, stats) { if (err) {throw err; } console.log(stats); var appender = startAppender(fd, stats.size); appender.append(new Buffer('append this!'), function(err) { console.log('appended'); }); }) });

File-system Exercises
You can e out the solutions at the end of this book.

Exercise 1 - get the size of a le

Having a le named a.txt, print the size of that les in bytes.

Exercise 2 - read a chunk from a le

Having a le named a.txt, print bytes 10 to 14.

Exercise 3 - read two chunks from a le

Having a le named a.txt, print bytes 5 to 9, and when done, read bytes 10 to 14.

Low-level le-system

45

Exercise 4 - Overwrite a le
Having a le named a.txt, Overwrite it with the UTF8-encoded string ABCDEFGHIJLKLMNOPQRSTUVXYZ0123456789abcdefghijklmnopqrstuvxyz.

Exercise 5 - append to a le
Having a le named a.txt, append utf8-encoded string abc to le a.txt.

Exercise 6 - change the content of a le

Having a le named a.txt, ange byte at pos 10 to the UTF8 value of 7.

HTTP
HTTP Server
You can easily create an HTTP server in Node. Here is the famous hp server Hello World example: Source in le: hp/hp_server_1.js
1 2 3 4 5 6 7 8 9

var http = require('http'); var server = http.createServer(); server.on('request', function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.write('Hello World!'); res.end(); }); server.listen(4000);

On line 1 we get the hp module, to whi we call createServer() (line 3) to create an HTTP server. We then listen for request type events, passing in a callba function that gets two arguments: the request object and the response object. We can then use the response object to write ba to the client. On line 5 we write a header (ContentType: text/plain) and the HTTP status 200 (OK). On line 6 we reply the string Hello World! and on line 7 we terminate the request. On line 9 we bind the server to the port 4000. So, if you run this script on node you can then point your browser to hp://localhost:4000 and you should see the Hello World! string on it. is example can be shortened to: Source in le: hp/hp_server_2.js
1 2 3 4

require('http').createServer(function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World!'); }).listen(4000);

Here we are giving up the intermediary variables for storing the hp module (since we only need to call it once) and the server (since we only need to make it listen on port 4000). Also, as a shortcut, the hp.createServer function accepts a callba function that will be invoked on every request. ere is one last shortcut here: the response.end function can accept a string or buer whi it will send to the client before ending the request. 46

HTTP

47

The http.ServerRequest object

When listening for request events, the callba gets one of these objects as the rst argument. is object contains:

req.url
e URL of the request, as a string. It does not contain the sema, hostname or port, but it contains everything aer that. You can try this to analyze the url: Source in le: hp/hp_server_3.js
1 2 3 4

require('http').createServer(function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(req.url); }).listen(4000);

and connect to port 4000 using a browser. Change the URL to see how it behaves.

req.method
is contains the HTTP method used on the request. It can be, for example, GET, POST, DELETE or any other one.

req.headers
is contains an object with a property for every HTTP header on the request. To analyze it you can run this server: Source in le: hp/hp_server_4.js
1 2 3 4 5 6

var util = require('util'); require('http').createServer(function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(util.inspect(req.headers)); }).listen(4000);

HTTP

48

and connect your browser to port 4000 to inspect the headers of your request. Here we are using util.inspect(), an utility function that can be used to analyze the properties of any object.

req.headers properties are on lower-case. For instance, if the browser sent a CaeControl: max-age: 0 header, req.headers will have a property named cae-control with the value max-age: 0 (this last one is untoued).

The http.ServerResponse object

e response object (the second argument for the request event callba function) is used to reply to the client. With it you can:

Write a header
You can use res.writeHead(status, headers), where headers is an object that contains a property for every header you want to send. An example: Source in le: hp/hp_server_5.js
1 2 3 4 5 6 7 8 9

var util = require('util'); require('http').createServer(function(req, res) { res.writeHead(200, { 'Content-Type': 'text/plain', 'Cache-Control': 'max-age=3600' }); res.end('Hello World!'); }).listen(4000);

On this example we set 2 headers: one with Content-Type: text/plain and another with CaeControl: max-age=3600. If you save the above source code into hp_server_5.js and run it with:

HTTP

49

$ node http_server_5.js

You can query it by using your browser or using a command-line HTTP client like curl:
1 2 3 4 5 6 7 8

$ curl -i http://localhost:4000 HTTP/1.1 200 OK Content-Type: text/plain Cache-Control: max-age=3600 Connection: keep-alive Transfer-Encoding: chunked Hello World!

Change or set a header

You can ange a header you already set or set a new one by using
1

res.setHeader(name, value);

is will only work if you havent already sent a piece of the body by using res.write().

Remove a header
You can remove a header you have already set by calling:
1

res.removeHeader(name, value);

Again, this will only work if you havent already sent a piece of the body by using res.write().

Write a piece of the response body

You can write a string:
1

res.write('Hello');

or a an existing buer:
1 2 3

var buf = new Buffer('Hello World'); buf[0] = 45; res.write(buffer);

is method can, as expected, be used to reply dynamically generated strings or binary le. Replying with binary data will be covered later.

HTTP

50

HTTP Client
You can issue hp requests using the hp module. Node is specically designed to be a server, but it

http.get()
Source in le: hp/hp_client_1.js is example uses hp.get to make an HTTP GET request to the url hp://www.google.com:80/index.html. You can try it by saving it to a le named hp_client_1.js and running:
1 2 3

$ node http_client_1.js got response: 302

http.request()
Using hp.request you can make any type of HTTP request:
1

http.request(options, callback);

e options are: * host: A domain name or IP address of the server to issue the request to. * port: Port of remote server. * method: A string specifying the HTTP request method. Possible values: GET (default), POST, PUT, and DELETE. * path: Request path. Should include query string and fragments if any. E.G. /index.html?page=12 * headers: An object containing request headers. e following method makes it easy to send body values (like when you are uploading a le or posting a form): Source in le: hp/hp_client_2.js
1 2 3 4 5 6 7 8

var options = { host: 'www.google.com', port: 80, path: '/upload', method: 'POST' }; var req = require('http').request(options, function(res) {

HTTP

51
console.log('STATUS: ' + res.statusCode); console.log('HEADERS: ' + JSON.stringify(res.headers)); res.setEncoding('utf8'); res.on('data', function (chunk) { console.log('BODY: ' + chunk); });

9 10 11 12 13 14 15 16 17 18 19 20

}); // write data to request body req.write("data\n"); req.write("data\n"); req.end();

On lines 18 and 19 we are writing the HTTP request body data (two lines with the data string) and on line 20 we are ending the request. Only then the server replies and the response callba gets activated (line 8). en we wait for the response. When it comes, we get a response event, whi we are listening to on the callba function that starts on line 8. By then we only have the HTTP status and headers ready, whi we print (lines 9 and 10). en we bind to data events (line 12). ese happen when we get a unk of the response body data (line 12). is meanism can be used to stream data from a server. As long as the server keeps sending body unks, we keep receiving them.

HTTP Exercises
You can eout the solutions at the end of this book.

Exercise 1
Make an HTTP server that serves les. e le path is provided in the URL like this: hp://localhost:4000/path/to/my/le.txt

Exercise 2
Make an HTTP server that outputs plain text with 100 new-line separated unix timestamps every second.

HTTP

52

Exercise 3
Make an HTTP server that saves the request body into a le.

Exercise 4
Make a script that accepts a le name as rst command line argument and uploads this le into the server built on the previous exercise.

Streams, pump and pipe

Node has a useful abstraction: Streams. More specically, two very useful abstractions: Read Streams and Write Streams. ey are implemented throughout several Node objects, and they represent inbound (ReadStream) or outbound (WriteStream) ow of data. We have already come across some of them, but here we will try to introduce them in a more formal way.

ReadStream
A ReadStream is like a faucet of data. Aer you have created a one - the method of creating them depends on the type of stream - , you can:

Wait for data

By binding to the data event you can be notied every time there is a unk being delivered by that stream. It can be delivered as a buer or as a string. If you use stream.setEncoding(encoding), the data events pass in strings. If you dont set an encoding, the data events pass in buers. So here is an example:
1 2 3 4 5 6 7 8 9 10

var readStream = ... readStream.on('data', function(data) { // data is a buffer; }); var readStream = ... readStream.setEncoding('utf8'); readStream.on('data', function(data) { // data is a utf8-encoded string; });

So here data passed in on the rst example is a buer, and the one passed on the second is a string because we are informing the stream about the encoding we are expecting.

e size of ea unk may vary, it may depend on buer size or on the amount of available data.

53

Streams, pump and pipe

54

Know when it ends

A stream can end, and you can know when that happens by binding to the end event like this:
1 2 3 4

var readStream = ... readStream.on('end', function() { console.log('the stream has ended'); });

Pause it
A read stream is like a faucet, and you can keep the data from coming in by pausing it like this:
1

readStream.pause();

Resume it
If its paused, the faucet can be reopened and the stream can start owing again:
1

readStream.resume();

WriteStream
A WriteStream is an abstraction on somewhere you can send data to. It can be a le or a network connection or even an object that outputs data that was transformed - like when zipping a le. With a WriteStream you can:

Write
You can write a buer or a string by calling write:
1 2

var writeStream = ...; writeStream.write('this is an utf8 string');

Here Node assumes we are passing an UTF-8-encoded string. Alternatively you can specify another encoding like this:
1 2

var writeStream = ...; writeStream.write('7e3e4acde5ad240a8ef5e731e644fbd1', 'base64');

Streams, pump and pipe

55

or you can simply write a buer:

1 2 3

var writeStream = ...; var buffer = new Buffer('this is a buffer with some string'); writeStream.write(buffer);

Wait for it to drain

Node does not blo on I/O, so it does not blo on read or write commands. On write commands, if Node is not able to ush the data into the kernel buers, it will buer that data for you, storing it in your process memory. Because of this, writeStream.write() returns a boolean. If write() manages to ush all data to the kernel buer, it returns true. If not, it returns false. When a writeStream manages to do ush the data into the kernel buers, it emits a drain event so you can listen to it like this:
1 2

var writeStream = ...; writeStream.on('drain', function() { console.log('drain emitted'); });

Later we will see how this draining notications combined with the pause and resume capabilities can come handy in limiting the memory growth of your Node process.

Some stream examples

Here are some instances of Node streams.

Filesystem streams
You can create a read stream for a le path by doing something like:
1 2 3 4 5

var fs = require('fs'); var rs = fs.createReadStream('/path/to/file'); ...

Here you can pass a second argument to fs.createReadStream where you can specify the start and end position on your le, the encoding, the ags and the buer size. Here are the defaults:

Streams, pump and pipe

56

1 2 3 4 5 6

{ flags: 'r', encoding: null, fd: null, mode: 0666, bufferSize: 64 * 1024 }

You can also create a write stream:

1 2 3

var fs = require('fs'); var rs = fs.createWriteStream('/path/to/file', options); ...

Whi also accepts a second argument with an options object. e options argument to createWriteStream has these default values:
1

{ flags: 'w',

encoding: null,

mode: 0666 }

For instance, to create a le WriteStream that assumes UTF-8 encoding you can use:
1 2

var fs = require('fs'); var rs = fs.createWriteStream('/path/to/file', { encoding: 'utf8' });

Network streams
ere are all kinds of streams on the networking API of Node. For instance, a client TCP connection is a write and a read stream. An hp request object is a read stream. An hp response object is a write stream. at is, ea implements the ReadStream / WriteStream methods and events.

The Slow Client Problem

As we said, Node does not blo on writes, and it buers the data for you if the write cannot be ushed into the kernel buers. Imagine this scenario: you are pumping data into a write stream (like a TCP connection to a browser), and your source of data is a read stream (like a le ReadStream):
1 2 3 4

require('http').createServer(function(req, res) { var rs = fs.createReadStream('/path/to/big/file'); rs.on('data', function(data) { res.write(data);

Streams, pump and pipe

57

5 6 7 8 9

}); rs.on('end', function() { res.end(); }); });

If the le is local, the read stream should be fast. If the connection to the client is slow, the writeStream will be slow. So readStream data events will happen quily, the data will be sent to the writeStream, but eventually Node will have to start buering the data because the kernel buers will be full. What will happen then is that the /path/to/big/le le will be buered in memory for ea request, and if you have many concurrent requests, Node memory consumption will inevitably increase, whi may lead to other problems, like swapping, thrashing and memory exhaustion.

What can we do?

To address this problem you will have to make use of the pause and resume of the read stream, and pace it alongside your write stream so your memory does not ll up:
1 2 3 4 5 6 7 8 9 10 11 12 13 14

require('http').createServer(function(req, res) { var rs = fs.createReadStream('/path/to/big/file'); rs.on('data', function(data) { if (!res.write(data)) { rs.pause(); } }); res.on('drain', function() { rs.resume(); }); rs.on('end', function() { res.end(); }); });

On line 5 we are pausing the readStream if the write cannot ush it to the kernel, and we are resuming it (line 9) when the writeStream is drained.

Pump
What was described here is a recurring paern, and instead of this complicated ain of events, you can simply use util.pump(), whi does exactly what we described:

Streams, pump and pipe

58

1 2 3 4 5 6 7 8

var util = require('util'); require('http').createServer(function(req, res) { var rs = fs.createReadStream('/path/to/big/file'); util.pump(rs, res, function() { res.end(); }); });

Mu simpler, right? util.pump accepts 3 arguments: the readable stream, the writable stream and a callba for when the read stream ends.

and his cousin pipe

A ReadStream can be piped into a WriteStream on the same fashion, simply by calling pipe(destination) like this:
1 2 3 4

require('http').createServer(function(req, res) { var rs = fs.createReadStream('/path/to/big/file'); rs.pipe(res); });

By default, end() is called on the destination when the read stream ends. You can prevent that behavior by passing in end: false on the second argument options object like this:
1 2 3 4 5 6 7

require('http').createServer(function(req, res) { var rs = fs.createReadStream('/path/to/big/file'); rs.pipe(res, {end: false}); rs.end(function() { res.end("And that's all folks!"); }); });

Making your own Of course you can implement your own read and write streams. ReadStream To sum it up, when creating a Readable stream, you have to implement the following methods:

Streams, pump and pipe

59

setEncoding(encoding) pause() resume() destroy() and emit the following events; data end error close fd (not mandatory) You should also implement the pipe() method, but you can lend some help from Node by inheriting from Stream like this:
1 2 3 4

var MyClass = ... var util = require('util'),; Stream = require('stream').Stream; util.inherits(MyClass, Stream);

is will make the pipe method available to you at no extra cost. WriteStream To implement your own WriteStream-ready pseudo-class you should provide the following methods: write(string, encoding=utf8, [fd]) write(buer) end() end(string, encoding) end(buer) destroy() and the emit the following events: drain error close

TCP
Node has a rst-class HTTP server implementation, but this server descends from the bare-bones TCP server. Being so, everything described here applies also to every class descending from the net.Server, like the hp.Server. You can create a TCP server using the net module like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

require('net').createServer(function(socket) { // new connection socket.on('data', function(data) { // got data }); socket.on('end', function(data) { // connection closed }); socket.write('Some string'); }).listen(4001);

On line 1 we use the createServer method on the net paage, whi we bind to TCP port 4001 on line 15. We can pass in a function callba to createServer to be invoked every time there is a connection event. On this soet object we can then listen to data events when we get a paage of data and the end event when that connection is closed. On a soet we can also:

Write a string or a buer

We can pass in a string or a buer to be sent through the soet. If a string is passed in, you can specify an encoding as a second argument like this:
1

var flushed = socket.write('453d9ea499aa8247a54c951', 'base64');

If you dont specify the encoding, Node will assume its UTF-8. 60

TCP

61

e soet object is an instance of net.Soet, whi is a writeStream, so the write method returns a boolean, saying whether it ushed to the kernel or not. You can also pass in a callba function to be invoked when the data is nally wrien out like this:
1 2 3

var flushed = connection.write('453d', 'base64', function() { // flushed });

or, assuming UTF-8:

1 2 3

var flushed = connection.write('I am UTF-8!', function() { // flushed });

end
You can end the connection by calling the end method. is sends the TCP FIN paet, notifying the other end that this end wants to close the connection. But you can still get data events aer you have issued this, simply because there still might be some data in transit, or the other end might be insisting on sending you some more data. Also, you can pass in some nal data to be sent when invoking end:
1

socket.end('Bye bye!');

and all the other methods

soet object is an instance of net.Soet, and it implements the writeStream and readStream interfaces, so all those methods are available, like pause() and resume(). Also you can bind to the drain events.

Idle sockets
You can also be notied when a soet has been idle for some time, i.e., there has been no data received. For that, you must dene a timeout by calling setTimeout():

TCP

62

1 2 3 4 5 6

var timeout = 60000; // 1 minute socket.setTimeout(timeout); socket.on('timeout', function() { socket.write('idle timeout, disconnecting, bye!'); socket.end(); });

or, in a shorter form:

1 2 3

socket.setTimeout(60000, function() { socket.end('idle timeout, disconnecting, bye!'); });

Keep-alive
In Node, a net.Soet can implement a keep-alive meanism to prevent timeouts occurring on the network or on the peer. Node does that by sending an empty TCP paet with the ACK ag turned on. You can enable the keep-alive functionality by:
1

socket.keepAlive(true);

You can also specify the delay between the last paet received and the next keep-alive paet on the second argument to the keepAlive call like this:
1

socket.keepAlive(true, 10000); // 10 seconds

Delay or no delay
When sending o TCP paets, the kernel buers data before sending it o, and uses the Naggle algorithm to determine when to send o the data. If you wish to turn this o and demand that the data gets sent immediately aer write commands, use:
1

socket.setNoDelay(true);

TCP

63

server.close()
is method closes the server, preventing it from accepting new connections. is function is asynronous, and the server will emit the close event when actually closed:
1 2 3 4 5

var server = ... server.close(); server.on('close', function() { console.log('server closed!'); });

Listening
As we saw, aer the server is created, we can bind it to a specic TCP port like this:
1 2 3

var port = 4001; var host = '0.0.0.0'; server.listen(port, host);

e second argument (host) is optional. If omied, the serve will accept connections directed to any IP address. is method is asynronous. To be notied when the server is really bound you have to pass a callba like this:
1 2 3

server.listen(port, host, function() { console.log('server listening on port ' + port); });

or without a host:
1 2 3

server.listen(port, function() { console.log('server listening on port ' + port); });

TCP client
You can connect to a TCP server using the net module like this:

TCP

64

1 2 3

var net = require('net'); var port = 4001; var conn = net.createConnection(port);

Here we omied the second argument for the createConnection function, whi is the host name. If you omit it, it defaults to localhost. Now with a host name:
1 2 3 4

var var var var

net = require('net'); port = 80; host = 'www.google.com'; conn = net.createConnection(port, host);

en you can listen for data:

1 2 3

conn.on('data', function(data) { console.log('some data has arrived') });

or send some data:

1

conn.write('some string over to you!');

or close it:
1

conn.close();

and also listen to the close event:

1 2 3

conn.on('close', function() { console.log('connection closed'); });

Soet conforms to the ReadStream and WriteStream interfaces, so you can use all of the previously described methods on it.

TCP

65

Error handling
When handling a soet on the client or the server you can (and should) handle the errors by listening to the error event like this:
1 2 3 4 5

require('net').createServer(function(socket) { socket.on('error', function(error) { // do something }); });

If you dont oose to cat an error, Node will handle an uncaught exception and terminate the current process. Unless you want that, you should handle the errors.

Also, you can oose to cat uncaught exceptions - preventing your Node process to go down - by doing something like:
1

process.on('uncaughtException', function() {

// Do something with it});

TCP Exercises
You can e the solutions at the end of the book.

Exercise 1
Make a at server that requires no authentication, just a TCP client connection. Ea time the client sends some text, the server boradcasts it to the other clients.

Exercise 2
Make a at client that accepts 2 command line arguments: host and port, and reads from stdin, sending data to the server on ea new line.

UNIX sockets
Server
e net.Server class not only supports TCP soets, it also supports UNIX domain soets. Unix domain soets are soets that are bound to the same host operating system. To create a UNIX soet server you have to create a normal net.Server, as you would on a TCP server, but then make it listen to a le path instead of a port like this:
1 2 3 4

var server = net.createServer(function(socket) { // got a client connection here... }); server.listen('/path/to/socket');

Unix domain soet servers present the exact same API as a TCP server.

Tip: If you are doing inter-process communication that is local to your host, consider using UNIX domain soets instead of TCP soets, as they should perform mu beer. For instance, when connecting node to a front-end web-server that stays on the same maine, oosing UNIX domain soets is generally preferable.

Client
To connect to a UNIX soet server, you can also use net.createConnection as when connecting to a TCP Server, but pass in a soet path instead of a port - like this:
1 2 3 4 5

var net = require('net'); var conn = net.createConnection('/path/to/socket'); conn.on('connect', function() { console.log('connected to unix socket server'); });

Here, on line 2, we are passing in a soet path instead of a port and a host name.

66

UNIX soets

67

Passing le descriptors around

UNIX soets have this interesting feature that allows you to pass le descriptors from a process into another process. In UNIX, a le descriptor can be a pointer to an open le or network connection, so this tenique can be used to share les and network connections between processes. For instance, to grab the le descriptor from a le read stream you should use the fd aribute like this:
1 2 3

var fs = require('fs'); var readStream = fs.createReadStream('/etc/passwd', {flags: 'r'}); var fileDescriptor = readStream.fd;

and then you can pass it into a UNIX soet using the second or third argument of soet.write like this:
1 2

var socket = ... socket.write('some string', fileDescriptor);

or, if you need to specify the string encoding:

1

socket.write('some string', 'utf8', fileDescriptor);

On the other end you can receive a le descriptor by listening to the fd event like this:
1 2 3 4

var socket = ... socket.on('fd', function(fileDescriptor) { // now I have a file descriptor });

Depending on the type of le descriptor you can use the Node API to:

Read or write into that le

If its a le-system le descriptor you can use the Node low level fs module API to read or write from / into it:

UNIX soets

68

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

var fs = require('fs'); var socket = ... socket.on('fd', function(fileDescriptor) { // write some var writeBuffer = new Buffer("here is my string"); fs.write(fileDescriptor, writeBuffer, 0, writeBuffer.length); // read some var readBuffer = new Buffer(1024); fs.read(fileDescriptor, readBuffer, 0, readBuffer.length, 0, function(err, bytesRead) { if (err) {console.log(err); return; } console.log('read ' + bytesRead + ' bytes:'); console.log(readBuffer.slice(0, bytesRead)); } ); });

Here, as you can see, we are reading and writing using the le descriptor that was brought to us from another process as if it was created locally.

You have to be careful, though, because that le descriptor must have been opened with the right ags. For instance, if you opened the le with the r ag, you should not be able to write into it independently of the process you are in.

Listen to the server socket

As another example on sharing a le descriptor between processes: if the le descriptor is a server soet that was passed in, you can create a server on the receiving end and associate the new le descriptor by using the server.listenFD method on it to it like this:
1 2 3 4 5 6 7

var server = require('http').createServer(function(req, res) { res.end('Hello World!'); }); var socket = ... socket.on('fd', function(fileDescriptor) { server.listenFD(fileDescriptor); });

UNIX soets

69

You can use listenFD() on an hp or net server, in fact, on anything that descends from net.Server.

Datagrams (UDP)
UDP is a connection-less protocol that does not provide the delivery aracteristics that TCP does. When sending UDP paets, you are not guaranteed of the order they might arrive in, and even if they will arrive at all. On the other hand, UDP can be quite useful in certain cases, like when you want to broadcast data, when you dont need hard delivery guarantees and sequence or even when you dont know the addresses of your peers.

Datagram server
You can setup a server listening on a UDP port like this: Code in: udp/udp_server_1.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

var dgram = require('dgram'); var server = dgram.createSocket('udp4'); server.on('message', function(message, rinfo) { console.log('server got message: ' + message + ' from ' + rinfo.address + ':' + rinfo.port); }); server.on('listening', function() { var address = server.address(); console.log('server listening on ' + address.address + ':' + address.port); }); server.bind(4000);

Here we are using the dgram module, whi provides a way to create a UDP soet (line 3). e createSoet function accepts the soet type as the rst argument , whi can be either udp4 (UDP over IPv4), udp6 (UDP over IPv6) or unix_dgram (UDP over unix domain soets). You can save this in a le named udp_server_1.js and run it:
1

$ node udp_server_1.js

70

Datagrams (UDP)

71

Whi should output the server address, port and wait for messages. You can test it using a tool like netcat ( hp://netcat.sourceforge.net/ ) like this:
1

$ echo 'hello' netcat -c -u -w 1 localhost 4000

is sends an UDP paet with hello to localhost port 4000. You should then get on the server output something like:
1 2

server got message: hello from 127.0.0.1:54950

Datagram client
To create an UDP client to send UDP paets you can do something like: Code in udp/udp_client_1.js
1 2 3 4 5 6 7

var dgram = require('dgram'); var client = dgram.createSocket('udp4'); var message = new Buffer('this is a message'); client.send(message, 0, message.length, 4000, 'localhost'); client.close();

Here we are creating a client using the same createSoet function we did to create the client, with the dierence that we dont bind.

You have to be careful not to ange the buer you pass on client.send before the message has been sent. If you need to know when your message has been ushed to the kernel, you should pass a last argument to client.send with a callba function to be invoked when the buer may be reused like this:
1 2 3

client.send(message, 0, message.length, 4000, 'localhost', function() { // you can reuse the buffer now });

http://netcat.sourceforge.net/

Datagrams (UDP)

72

Since we are not binding, the message is sent from a UDP random port. If we wanted to send from a specc port, we could have used client.bind(port) like this: Code in: udp/udp_client_2.js
1 2 3 4 5 6 7 8

var dgram = require('dgram'); var client = dgram.createSocket('udp4'); var message = new Buffer('this is a message'); client.bind(4001); client.send(message, 0, message.length, 4000, 'localhost'); client.close();

Here we are binding to the specic port 4001, and when saving this le and executing it, the running server should output something like:
1

server got message: this is a message from 127.0.0.1:4001

is port binding on the client really mixes what a server and a client are, and can be useful for maintaining conversations like this:
1 2 3 4 5 6 7 8 9 10 11

var dgram = require('dgram'); var client = dgram.createSocket('udp4'); var message = new Buffer('this is a message'); client.bind(4001); client.send(message, 0, message.length, 4000, 'localhost'); client.on('message', function(message, rinfo) { console.log('and got the response: ' + message); client.close(); });

Here we are sending a message, and also listening to messages. When we receive one message we close the client. Dont forget that UDP is unreliable, and whatever protocol you devise on top of it, account that messages can be lost and delivered out of order!

Datagrams (UDP)

73

Datagram Multicast
One of the interesting uses of UDP is to distribute messages to several nodes using only one network message. is can have manu uses like doing logging, cae cleaning and generally on the cases where you can aord to loose some messages.

Message multicasting can be useful when you dont want to need to know the address of all peers. Peers just have to tune in and listen to that annel.

Nodes can report their interest in listening to certain multicast annels by tuning into that annel. In IP addressing there is a space reserved for multicast addresses. In IPv4 the range is between 224.0.0.0 and 239.255.255.255, but some of these are reserved. 224.0.0.0 through 224.0.0.255 is reserved for local purposes (as administrative and maintenance tasks) and the range 239.0.0.0 to 239.255.255.255 has also been reserved for administrative scoping.

Receiving multicast messages

To join a multicast address like 230.1.2.3 you can do something like this: Code in: udp/udp_multicast_listen.js
1 2 3 4 5 6 7 8 9

var server = require('dgram').createSocket('udp4'); server.on('message', function(message, rinfo) { console.log('server got message: ' + message + ' from ' + rinfo.address + ':' + rinfo.port); }); server.addMembership('230.1.2.3'); server.bind(4000);

On line 7 we are saying to the kernel that this UDP soet should receive multicast messages for the multicast address 230.1.2.3. When calling the addMembership you can pass the listening interface as an optional second argument. If omied, Node will try to listen on every public interface. en you can test the server using netcat like this:
1

$ echo 'hello' netcat -c -u -w 1 230.1.2.3 4000

Datagrams (UDP)

74

Sending multicast messages

To send a multicast message you simply have to specify the multicast address like this:
1 2 3 4 5 6 7 8

var dgram = require('dgram'); var client = dgram.createSocket('udp4'); var message = new Buffer('this is a multicast message'); client.setMulticastTTL(10); client.send(message, 0, message.length, 4000, '230.1.2.3'); client.close();

Here, besides sending the message, we previously set the Multicast time-to-live to 10 (an arbitrary value here). is TTL tells the network how many hops (routers) it can travel through before it is discarded. Every time a UDP paet travels through a hop, the TTL counter is decremented, and if 0 is reaed, the paet is discarded.

What can be the datagram maximum size? really depends on the network it travels through. e UDP header allows up to 65535 bytes of data, but if you are sendingpaet across an Ethernet network, for instance, the Ethernet MTU is 1500 bytes, limiting the maximum datagram size. Also, some routers will aempt to fragmentlarge UDP paet into 512 byte unks.

UDP Exercises
Exercise 1
Create a UDP server that eoes the messages ot receives ba into the origin soet.

Child processes
On Node you can spawn ild processes, whi can be another Node process or any process you can laun from the command line. For that you will have to provide the command and arguments to execute it. You can either spawn and live along side with the process (spawn), or you can wait until it exits (exec).

Executing commands
You can then laun another process and wait for it to nish like this:
1 2 3 4 5 6 7 8 9

var exec = require('child_process').exec; exec('cat *.js wc -l', function(err, stdout, stderr) { if (err) { console.log('child process exited with error code ' + err.code); return; } console.log(stdout); });

Here on line 3 we are passing in cat *.js wc -l as the command, the rst argument to the exec invokation. We are then passing as the second argument a callba function that will be invoked once the exec has nished. If the ild process returned an error code, the rst argument of the callba will contain an instance of Error, with the code property set to the ild exit code. If not, the output of stdout and stderr will be collected and be oered to us as strings. You can also pass an optional options argument between the command and the callba function like this:
1 2

var options = {timeout: 10000}; exec('cat *.js wc -l', options, function(err, stdout, stderr) {

...});

e available options are: encoding : the expected encoding for the ild output. Defaults to utf8; timeout : the timeout in milliseconds for the execution of the command. Defaults to 0, whi does not timeout; 75

Child processes

76

maxBuer : species the maximum size of the output allowed on stdout or stderr. If exceeded, the ild is killed. Defaults to 200 * 1024; killSignal : the signal to be sent to the ild if it times out or exceeds the output buers. Identied as a string; cwd : current working directory; env: environment variables to be passed into the ild process. Defaults to null.

On the killSignal option you can pass a string identifying the name of the signal you wish to send to the target process. Signals are identied in node as strings. For a complete list of strings type on your shell:
1

$ man signal

Scroll down, and you will see a list of constants representing the signals. ose are the strings used in Node.

Spawning processes
You can spawn a new ild process based on the ild_process.spawn function like this:
1 2 3 4 5 6

var spawn = require('child_process').spawn; var child = spawn('tail', ['-f', '/var/log/system.log']); child.stdout.on('data', function(data) { console.log('stdout: ' + data); });

Here we are spawning a ild process to run the tail command, passing in as arguments -f and /var/log/system.log. is tail command will monitor the le /var/log/system.log - if it exists and output every new data appended to it into the stdout. On line 4 we are listening to the ild stdout and printing its output. So here, in this case, we are piping the anges to the /var/log/system.log le into our Node application. Besides the stdout we can also listen to the stderr ild output stream like this:
1 2 3

child.stderr.on('data', function(data) { console.log('stderr: ' + data); });

Child processes

77

Killing processes
You can (and should) eventually kill ild processes by calling the kill method on the ild object:
1 2 3 4 5 6

var spawn = require('child_process').spawn; var child = spawn('tail', ['-f', '/var/log/system.log']); child.stdout.on('data', function(data) { console.log('stdout: ' + data); child.kill(); });

is sends a SIGTERM signal to the ild process. You can also send another signal to the ild process. You need to specify it inside the kill call like this:
1

child.kill('SIGKILL');

Child Processes Exercises

Exercise 1
Create a server that a) opens a le b) listens on a unix domain soet and c) spawns a client. is client opens the soet to the server and waits for a le descriptor. e server then passes in the le descriptor we opened on a). e client writes to the le and quits. When the client process quits, the server quits.

Streaming HTTP chunked responses

One of the great features of Node is to be extremely streamable, and being HTTP a rst-class protocol in Node, HTTP responses are no dierent. HTTP unked encoding allows a server to keep sending data to the client without ever sending the body size. Unless you specify a Content-Length header, Node HTTP server sends the header
1

Transfer-Encoding: chunked

to the client, whi makes it wait for a nal unk with length of 0 before giving the response as terminated. is can be useful for streaming data - text, audio, video - or any other into the HTTP client.

A streaming example
Here we are going to code an example that pipes the output of a ild process into the client: Source code in: apters/unked/unked.js
1 2 3 4 5 6 7 8 9

var spawn = require('child_process').spawn; require('http').createServer(function(req, res) { var child = spawn('tail', ['-f', '/var/log/system.log']); child.stdout.pipe(res); res.on('end', function() { child.kill(); }); }).listen(4000);

Here we are creating an HTTP server (line 3) and binding it to port 4000 (line 9). When there is a new request we laun a new ild process by executing the command tail -f /var/log/system.log (line 4) whi output is being piped into the response (line 5). When the response ends (because the browser window was closed, or the network connection was severed, for instance), we kill the ild process so it does not hang around aerwards indenitely. So here, in 9 lines of code, we are making a Node streaming server that spawns, pipes the output of a process and then kills it as needed. 78

Streaming HTTP unked responses

79

Streaming Exercises
Exercise 1
Create a mixed TCP and HTTP server that, for every HTTP request, streams all the TCP clients input into the request response.

TLS / SSL
TLS (Transport Layer Security) and SSL (Secure Soet Layer) allow client / server applications to communicate across a network in a way designed to prevent eavesdropping (others looking into your messages) and tampering (others anging your message). TLS and SSL encrypt the segments of network connections above the Transport layer, enabling both privacy and message authentication. TLS is a standard based on the earlier SSL specications developed by Netscape. In fact, TLS 1.0 is also known as SSL 3.1 , and the latest version (TLS 1.2) is also known as SSL 3.3. So, from hereon, we will be using TLS instead of the deprecated SSL.

Public / private keys

Node TLS implementation is based on the OpenSSL library. Chances are you have the library installed, also as the openssl command-line utility. If its not installed, you should be looking for a paage named openssl.

Private key
TLS is a public / private key infrastructure. Ea client and server must have a private key. A private key can be created by the openssl utility on the command line like this:
1

$ openssl genrsa -out my.pem 1024

is should create a le named my.pem with your private key.

Public key
All servers and some clients need to have a certicate. Certicates are public keys signed by a Certicate Authority or self-signed. e rst step to geing a certicate is to create a Certicate Signing Request (CSR) le. is can be done with:
1

$ openssl req -new -key my_key.pem -out my_csr.pem

is will create a CSR le named my_csr.pem. To create a self-signed certicate with the CSR, you can do this:
1

openssl x509 -req -in my_csr.pem -signkey my_key.pem -out my_cert.pem

is will create a self-signed certicate le named my_cert.pem. Alternatively you can send the CSR to a Certicate Authority for signing. 80

TLS / SSL

81

TLS Client
You can connect to a TLS server using something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

var tls = require('tls'), fs = require('fs'), port = 3000, host = 'myhost.com', options = { key : fs.readFileSync('/path/to/my/private_key.pem'), cert : fs.readFileSync('/path/to/my/certificate.pem') }; var client = tls.connect(port, host, options, function() { console.log('connected'); console.log('authorized: ' + client.authorized); client.on('data', function(data) { client.write(data);// just send data back to server }); });

First we need to inform Node of the client private key and client certicate, whi should be strings. We are then reading the pem les into memory using the synronous version of fs.readFile, fs.readFileSync.

- Here we are using fs.readFileSync, a synronous function. Wont this blo the event loop? No, this will just run on the initialization of our app. As long as you dont us bloing functions inside an event handler, you should be ok. - Wait, what if this is a module we are requiring this inside a callba? You shouldnt be requiring modules inside callbas. ey do synronous le system access and will blo your event loop.

en, on line 10 we are connecting to the server. tls.connect returns a CryptoStream object, whi you can use normally as a ReadStream and WriteStream. On line 13 we just wait for data from the server as we would on a ReadStream, and then we, in this case, send it ba to the server on line 14.

TLS / SSL

82

TLS Server
A TLS server is a subclass of net.Server. With it you can make everything you can with a net.Server, except that you are doing over a secure connection. Here is an example of a simple eo TLS server:
1 2 3 4 5 6 7 8 9 10

var tls = require('tls'); fs = require('fs'); options = { key : fs.readFileSync('/path/to/my/server_private_key.pem'), cert : fs.readFileSync('/path/to/my/server_certificate.pem') }; tls.createServer(options, function(s) { s.pipe(s); }).listen(4000);

Besides the key and cert options, tls.createServer also accepts: requestCert : If true the server will request a certicate from clients that connect and aempt to verify that certicate. Default: false. rejectUnauthorized : If true the server will reject any connection whi is not authorized with the list of supplied CAs. is option only has an eect if requestCert is true. Default: false.

Verication
On both the client and the server APIs, the stream has a property named authorized. is is a boolean indicating if the client was veried by one of the certicate authorities you are using, or one that they delegate to. If s.authorized is false, then s.authorizationError contains the description of how the authorization failed.

TLS Exercises
Exercise 1
Create a certicate authority. Create a client certicate signed by this new certicate authority.

Exercise 2
Create a TLS eo server that uses the default certicate authorities.

TLS / SSL

83

Exercise 3
Create a TLS client that reads from stdin and sends it to the eo TLS server created on exercise 2.

Exercise 4
Make the TLS server only accept connections if the client is certied. Verify that he does not let the client created on exercise 3 connect.

Exercise 5
Make the TLS Server use the same certicate authority you used to sign the client certicate with. Verify that the server now accepts connections from this client.

HTTPS
HTTPS is the HTTP protocol over TLS. In Node HTTPS is implemented as a separate module. e HTTPS API is very similar to the HTTP one, with some honorable small dierences.

HTTPS Server
To create a server, you can do something like this:
1 2 3 4 5 6 7 8 9 10 11 12

var https = require('https'), fs = require('fs'); var options = { key: fs.readFileSync('/path/to/server/private_key.pem'), cert: fs.readFileSync('/path/to/server/cert.pem') }; https.createServer(options, function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World!'); });

So here, the rst argument to hps.createServer is an options object that, mu like in the TLS module, provides the private key and the certicate strings.

HTTPS Client
To make a HTTPS request you must also use the hps module like this:
1 2 3 4 5 6 7 8 9

var https = require('https'); var options = { host: 'encrypted.google.com', port: 443, path: '/', method: 'GET' }; var req = https.request(options, function(res) {

84

HTTPS

85

10 11 12 13 14 15 16 17

console.log("statusCode: ", res.statusCode); console.log("headers: ", res.headers); res.on('data', function(d) { process.stdout.write(d); }); }); req.end();

Here the options object, besides the hp.request options, also accepts: port: port of host to request to. Defaults to 443. key: e lient private key string to use for SSL. Defaults to null. cert: e client certicate to use. Defaults to null. ca: An authority certicate or array of authority certicates to e the remote host against. If may want to use the key and cert options if the server needs to verify the client. Also, you may pass the ca argument, whi is a certicate authority certicate or an array of them with whi you may verify the server against. Mu like the hp module, this module also oers a shortcut hps.get method that can be used like this:
1 2 3 4 5

var https = require('https'); var options = { host: 'encrypted.google.com', path: '/' }; https.get(options, function(res) { res.on('data', function(d) { process.console.log(d.toString()); });

Making modules
CommonJS modules
When craing your rst Node app, you tend to cram everything into one le, but sooner or later you will need to expand. e Node way to spread your app is to compartmentalize it in logic blos called modules. ese modules will have an interface, exposing module properties like functions or simple aributes.

One le module
To create a module you simply have to create a le somewhere in your app dir tree (lib/my_module.js is perhaps the appropriate place to start). Inside ea module you can use the global namespace without fear of stepping on another modules toes. And, at the end, you expose only what you wish to expose by assigning it to module.exports. Here is a qui example:
1 2 3 4 5 6 7 8 9 10 11

var counter = 0; var onePrivateMethod = function() { return counter; } ; var onePublicMethod = function() { onePrivateMethod(); return 'you already called this module ' + counter + ' times'; }; module.exports = onePublicMethod;

Here we are exporting (on the last line) only one function. If we save this module in the current directory under my_module.js:
1 2 3

var myModule = require('./my_module'); myModule(); // => 'you already called this module 1 times';

You can export any Javascript object you wish, so, for instance, you can export an object that has a collection of functions like this: 86

Making modules

87

1 2 3 4 5 6 7 8 9 10 11 12 13 14

var counter1 = 0; var onePublicMethod = function() { return 'you already called this function ' + (++ counter1) + ' times'; }; var counter2 = 0; var anotherPublicMethod = function() { return 'you already called this function ' + (++ counter2) + ' times'; } module.exports = { functionA: onePublicMethod, functionB: anotherPublicMethod };

A client using this module would look something like:

1 2 3 4 5 6 7

var myModule = require('./my_module'); myModule.functionA(); // => 'you already called this function 1 times'; myModule.functionA(); // => 'you already called this function 2 times'; myModule.functionB(); // => 'you already called this function 1 times';

An aggregating module
Also, modules can aggregate other modules and mix and expose them as they wish. For instance, su a module could look like:
1 2 3 4 5 6 7 8 9 10 11 12

var moduleA = require('./moduleA'); var moduleB = require('./moduleB'); var myFunc = function() { return "doing some crazy stuff"; } module.exports = { funcA: moduleA.funcA, funcB: moduleB.funcB, funcC: myFunc }

Making modules

88

A pseudo-class

If you need to learn about Javascript pseudo-classes and prototypical inheritance I can recommend that you read the book Eloquent Javascript or Douglas Crofords Javascript - e Good Parts.

It is possible to implement a classlike(ish) behavior on your module using something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

var Line = function(x1, y1, x2, y2) { this.x1 = x1; this.y1 = y1; this.x2 = x2; this.y2 = x2; }; Line.prototype.length = function() { return Math.sqrt( Math.pow(Math.abs(this.x1 - this.x2), 2) + Math.pow(Math.abs(this.y1 - this.y2), 2) ); }; module.exports.create = function(x1, y1, x2, y2) { return new Line(x1, y1, x2, y2); };

Here we are creating the Line pseudo-class, but we are not exporting its constructor directly. Instead, we are exporting a create function, whi calls the constructor for us. We are doing this because people using this module may not remember that it would be necessary to use the new keyword when invoking the constructor function. If they forgot to do so, this would be bound to the global namespace, yielding very strange result. To prevent it we just export one create function, leading to clear module usage like:
1 2 3

var Line = require('./line'); var line = Line.create(2, 4, 10, 15); console.log('this line length is ' + line.length());
http://eloquentjavascript.net/

Making modules

89

A pseudo-class that inherits

Besides implementing a class behavior, you can also inherit it from another class. For instance, the EventEmier class is a very useful to use like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

var util = require('util'), EventEmitter = require('events').EventEmitter; var Line = function(x1, y1, x2, y2) { this.x1 = x1; this.y1 = y1; this.x2 = x2; this.y2 = x2; }; util.inherits(Line, EventEmitter); Line.prototype.length = function() { return Math.sqrt( Math.pow(Math.abs(this.x1 - this.x2), 2) + Math.pow(Math.abs(this.y1 - this.y2), 2) ); };

Note that you should call util.inherits before declaring the prototype properties on the pseudo-class - like the Line.prototype.length on the previous example. If you call util.inherits aer, they will be removed, since the prototype object is replaced on line 11.

Now you should be ready to create your own modules so your app code doesnt have to live on a single le!

node_modules and npm bundle

As explained on the module loading apter, Node tries to use the nearest node_modules directory by batraing the current directory up to root. What this means is that, generally, you can put the external modules your application depends on into a node_modules folder inside your app root folder, thus bundling and freezing your application dependencies.

Making modules

90

Fortunately npm can do that for you. For that to work you need to declare your application dependencies inside a paage.json le like this: Source code in apters/paaging/app1
1 2 3 4 5 6 7 8 9 10

{ , , , ,

"name" : "app1" "version" : "0.1.0" "description" : "Hands-on packaging app example" "main" : "app.js" "dependencies" : { "express" : ">= 1.0" , "jade" : "0.8.5" }

is is a minimal paage.json, you can add more information to it. You can type
1

$ npm help json

to get a man page documenting the JSON document format.

On line 7 and 8 we declare that this application depends on express and jade npm paages, specifying the version needs.

Bundling
Having now the paage.json paage in your root directory you can bundle all your dependencies into a node_modules by typing into your console, in your app root dir:
1

$ npm bundle

is will create a node_modules inside the app root dir, and when your app is executed, node will rst look into this directory to resolve modules rst.

Using this tenique you can paage an application so you dont run into cross-dependencies problems on co-existing applications and removing the need to install paages globally.

Debugging
If you nd yourself in a situation where you need to inspect the inner workings of your Node app code, there are several tools that can come to your aid.

console.log
e simplest one is console.log. You can use it to inspect objects like this:
1 2

var obj = {a: 1, b: 2}; console.log(obj);

Whi will print

1

{ a: 1, b: 2 }

Node built-in debugger

If you need to halt execution to carefully inspect your app, you can use Node built-in debugger. Node has a built-in debugger that is a simple and basic one, but can be just enough to help you debug your app. First, you have to insert an initial breakpoint in your app by inserting a debugger instruction like this:
1 2 3

var a = 1; debugger; var b = a + 1;

And them you can start the Node debuuger on your app like this:
1

$ node debug my_app.js

is will laun the Node debugger. You should now get the debugger prompt, but your app is not running just yet. Inside the debugger prompt you will have to type:
1

$ run

91

Debugging

92

And your app will start running, hiing your rst debugger instruction. It will then stop on the breakpoint you set as soon as it is encountered. If you type
1

$ list

you will see where you are inside your script. You can inspect the local scope. You can print variables like this:
1

$ print a

If you type
1

$ next

you will step into the next line of your script. If you type
1

$ continue

your app will resume, stopping if it passes another (or the same) breakpoint. When inside the prompt you can kill your app by commanding:
1

$ kill

Node Inspector
Another debugging tool is Node Inspector. is debugger brings the full-edged Chrome inspector to your Node app. You can install Node Inspector like this:
1

$ npm install node-inspector

Node Inspector runs as a daemon by default on port 8080. You can laun it like this:
1

$ node-inspector &

Debugging

93

is will send the node-inspector process to the baground. Next you need to re up your app, but using a debug or debug-brk option on the node executable like this:
1

$ node --debug-brk myapp.js

e debug-brk option will make your app break on the rst line, while the debug option will simply enable debugging.

Tip: when debugging servers you will want to use debug, and when debugging other scripts you may want to break on the rst line by using debug-brk.

Now you can open your browser and point it to hp://localhost:8080, and you should get something like this:

Debugging

94

You can set and unset breakpoints by cliing on the line numbers. When a breakpoint is reaed, your app freezes and Node Inspector shows the context:

Debugging

95

You can see the two most interesting context panes on the right: e call sta, where you can see whi functions were invoked to get to this point. e scope variables, where you can inspect the local variables, the global variables and the closure variables, whi are variables that are dened on a higher function scope. Above those panes you can see some buons whi you can use to manipulate the executed instructions: Continue execution up until a breakpoint is reaed. Execute this function and stop on the next line. Step into this function. Continue, and when this function ends return to the caller.

Debugging

96

Live edit
You can ange the code while you are debugging. For this you can double-cli on a line of code and edit it. Change the code and hit the Tab key or cli outside the edit area. And voila, you just anged running code!

Changing the code like this will not save save the anges - it should only be used to test qui xes.

Automated Unit Testing

When doing a module you should also create an automated way to test the module. A set of tests that test a module are called unit tests. Lets say you want to create the unit tests for a module that exports a function that sums two integers:
1 2 3

module.exports = function(a, b) { return a + b; };

For that you need two tools: a test runner and an assertion module.

A test runner
A test runner is a piece of soware that loads your tests, runs them and then presents the test result. e result can be positive (test passed) or negative (test failed), generally accompanied by a description of the failure. ere are various test runners, and my favorite nowadays is expresso. You can install it by running:
1

$ npm install expresso

On your app you should create a tests dir under the root, and create a le module for ea one of your modules. So, we would create a test module under tests/sum.js. is module should export an object that contains one property per test you wish to run. Ea one of these properties is a function that is launed asynronously. When all are done or failed, expresso reports how many were run and how many failed / succeeded. A test le for the sum module would be something like this:
1 2 3 4 5 6 7

var sum = require('../lib/sum'); module.exports.test1 = function() { // one test here }; module.exports.test2 = function() { // another test here };

To run expresso on all les inside the tests directory you can invoke the expresso executable like this: 97

Automated Unit Testing

98

$ expresso test/*.js

You can obtain more info about expresso here hp://visionmedia.github.com/expresso/.

Assertion testing module

An assertion testing module is something that allows you to easily compare results to what you expect. For instance, a module that does the sum of two integers should be compared against some known cases. Node already comes with a basic assertion module named assert. You can use the following module functions: assert.equal(a, b, [message]) - test shallow equality (with == ); assert.deepEqual(a, b, [message]) - test for deep equality; assert.notEqual(a, b, [message]) - test shallow inequality (as with !=); assert.notDeepEqual(a, b, [message]) - test for deep inequality; assert.strictEqual(a, b, [message]) - test strict equality (as with ===); assert.notStrictEqual(a, b, [message]) - test strict inequality (as with !==); assert.throws(blo, [error], [message]) - test if the blo (given as function) throws an error. With the optional error argument you can pass in an instance to be compared with the thrown exception, a regular expression to be compared with the exception, or a validation function. assert.doesNotrow(blo, [error], [message]) - the negation of the result of assert.throws() with the same arguments. All these messages have a nal optional argument where you can pass the message in case of failure.
http://visionmedia.github.com/expresso/

Automated Unit Testing

99

should.js
Another useful assertion testing module is should.js. Should.js provides a nice extension to the objects, into whi you can perform tests using a nice API that you can ain like this:
1

value.should.be.a('object').and.have.property('name', 'Pedro');

You can install Should.js using npm like this;

1

$ npm install should

and then include the module

1

require('should');

ere is no need to assign a variable to the module since should extends With it you can: Assert truthfulness:
1 2 3

true.should.be.ok 'yay'.should.be.ok

or untruthfulness:
1 2 3

false.should.not.be.ok ''.should.not.be.ok

=== true
1 2 3

true.should.be.true '1'.should.not.be.true

=== false
1 2 3

false.should.be.false ''.should.not.be.false

Automated Unit Testing

100

emptiness
1 2 3

[].should.be.empty ''.should.be.empty

equality
1 2 3

({ foo: 'bar' }).should.eql({ foo: 'bar' }) [1,2,3].should.eql([1,2,3])

equal (strict equality)

1 2 3 4 5

(4).should.equal(4) 'test'.should.equal('test') [1,2,3].should.not.equal([1,2,3])

assert numeric range (inclusive) with within

1

.should.be.within(10, 20);

test numeric value is above given value:

1 2 3

.should.be.above(5) .should.not.be.above(15)

test numeric value is below given value:

1 2 3

.should.not.be.below(5) .should.be.below(15)

mating rec+gular expressions

1

"562".should.match(/[0-9]{3}/)

Automated Unit Testing

101

test length
1

[1, 2, 3].should.have.length(3)

substring inclusion
1

"abcdef".should.include.string('bc')

assert typeof
1 2 3

{a:1, b:2}.should.be.a('object') "test".should.be.a('string')

property existence
1 2 3

{a:1, b:2}.should.have.property('a'); {a:1, b:2}.should.not.have.property('c');

array containment
1 2 3

[1,2,3].should.contain(3) [1,2,3].should.not.contain(4)

own object keys

1 2 3

var obj = { foo: 'bar', baz: 'raz' }; obj.should.have.keys('foo', 'bar'); obj.should.have.keys(['foo', 'bar']);

responds to, asserting that a given property is a function:

1

user.should.respondTo('email')

Automated Unit Testing

102

Putting it all together

e source code for this section can be found at: apters/testing So now we should be able to make a test for our sum module:
1 2 3 4 5 6 7 8 9 10 11 12 13 14

require('should'); var sum = require('../lib/sum'); module.exports.testSumToZero = function() { sum(0, 5).should.equal(5); }; module.exports.testSumToZero2 = function() { sum(5, 0).should.equal(5); }; module.exports.someSums = function() { sum(1, 1).should.equal(2); sum(1, 2).should.equal(3); sum(2, 1).should.equal(3); sum(10, 120).should.equal(130); };

And then we can run our test from the command line like this;
1

$ expresso tests/*.js

Whi should print out:

1

% 3 tests

In case you are testing callbas, you can also use a rst argument given to all testing functions, whi we can name beforeExit. is function can be used to aa callbas before the tests end, so you can test if your callbas have been called or not:
1 2 3 4 5 6

module.exports.testAsync = function(beforeExit) { var n = 0; setTimeout(function(){ ++n; assert.ok(true); }, 200);

Automated Unit Testing

103

7 8 9 10 11 12 13 14

setTimeout(function(){ ++n; assert.ok(true); }, 200); beforeExit(function(){ assert.equal(2, n, 'Ensure both timeouts are called'); }); };

Callback ow
As you may have noticed, asynronous programming does not rely on the sta to organize ow between caller and called function. Instead, it relies on callba functions that are usually passed as arguments. Imagine that you would have to build a script that does the following: Append bytes 10-20 from le a.txt into le b.txt. Both les already exist. A solution may be something like this: Source code in: ow/exercise_1.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

var fs = require('fs'); var doWhatWasAsked = function(callback) { fs.open(__dirname + '/a.txt', 'r', function(err, aFd) { if (err) { callback(err); return; } var buffer = new Buffer(10); fs.read(aFd, buffer, 0, 10, 10, function(err, bytesRead) { if (err) { callback(err); return; } fs.open(__dirname + '/b.txt', 'a', function(err, bFd) { if (err) { callback(err); return; } fs.fstat(bFd, function(err, bStats) { if (err) { callback(err); return; } fs.write(bFd, buffer, 0, 10, bStats.size, callback); }) }) }); }) }; console.log('starting...'); doWhatWasAsked(function(err) { if (err) { throw err; } console.log('done'); });

Here we devise a function sillily called doWhatWasAsked, whi receives a callba to be invoked when there is an error or when the task is done. is function opens a.txt (line 4), and then reads 10 bytes starting at pos 10 (line 7). en it opens b.txt (line 10), es its size using fs.stat (line 12) and then writes into the end of the le (line 14). 104

Callba ow

105

The boomerang eect

In this example you can see what can be called a boomerang eect of callba aining, where the text indentation increases and then decreases along with function nesting. is can turn your code into callba spaghei, making it visually hard to tra whi context you are in. is style also makes debugging your application dicult, reducing even more the maintainability of your code. ere is more than one way around this. One is: instead of using anonymous inline functions we can use named functions - like this: Source code in ow/exercise_2.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

var fs

= require('fs');

var doWhatWasAsked = function(callback) { var aFd, bFd, buffer = new Buffer(10); function openA() { fs.open(__dirname + '/a.txt', 'r', readFromA); }; function readFromA(err, fd) { if (err) { callback(err); return; } aFd = fd; fs.read(aFd, buffer, 0, 10, 10, openB); } function openB(err) { if (err) { callback(err); return; } fs.open(__dirname + '/b.txt', 'a', statB); } function statB(err, fd) { if (err) { callback(err); return; } bFd = fd; fs.fstat(bFd, writeB); }; function writeB(err, bStats) { if (err) { callback(err); return; } fs.write(bFd, buffer, 0, 10, bStats.size, callback); }

Callba ow

106

30 31 32 33 34 35 36 37

openA(); }; console.log('starting...'); doWhatWasAsked(function(err) { if (err) { throw err; } console.log('done'); });

is code does what the previous code did, but its unarguably clearer. We are now declaring one named function for ea callba all under the same scope, and using the function names to pass them as the next callba to be executed. e downside of this tenique is that we loose the closure scopes, so we need to store the application state on a common scope (line 4). We could discuss whi approa is more elegant, but this one is certainly more readable.

Step
ere also are some tools to prevent the boomerang eect, and one of them is called Step. Step is a way to easily ain callba functions that respect a convention. To install Step you can:
1

$ npm install step

Step exports one function whi is the Step module itself. is function accepts a variable number of arguments, and ea one of them has to be a function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14

var Step = require('step'); Step ( function doA() { //... }, function doB(err, arg1, arg2) { //... }, function doC(err, arg1) { //... }, ... );

Callba ow

107

e rst argument of ea function is reserved for an error object. is convention follows the Node standard, . Any exceptions thrown are caught and passed as the rst argument to the next function. Also, the remaining arguments passed into the callba are passed into the next function untoued. Here we are naming the functions, nd you should do too to make the code clearer, but its not mandatory. Instead of passing a callba into the asynronous functions, you pass the this object. So our task would look something like this: Source code in ow/exercise_3.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

var fs = require('fs'), Step = require('step'); var doWhatWasAsked = function(callback) { var aFd, bFd, buffer = new Buffer(10); Step( function openA() { fs.open(__dirname + '/a.txt', 'r', this); }, function readFromA(err, fd) { if (err) { callback(err); return; } aFd = fd; fs.read(aFd, buffer, 0, 10, 10, this); }, function openB(err) { if (err) { callback(err); return; } fs.open(__dirname + '/b.txt', 'a', this); }, function statB(err, fd) { if (err) { callback(err); return; } bFd = fd; fs.fstat(bFd, this); }, function writeB(err, bStats) { if (err) { callback(err); return; } fs.write(bFd, buffer, 0, 10, bStats.size, callback); } ); }; console.log('starting...');

Callba ow

108

32 33 34 35

doWhatWasAsked(function(err) { if (err) { throw err; } console.log('done'); });

So here we are using Step this object as a callba (except on the writeB function, whi is the last in the ain, where we pass in the callba that was passed in the doWhatWasAsked function, terminating the request).

e downside of this approa is that we cannot use the power of closures. We cant address an argument that was passed in an earlier callba or a variable that was created inside a higher closure closure. We have to store them on a common global scope, like we do with the variables aFd, bFd and buer.

Parallel execution
Step also allows you to execute I/O in parallel by passing this.parallel() instead of this. In this case, the next callba in the ain will only be executed once all the parallel calls on the previous one have nished executing. e next callba will get an error on the rst argument that has an Error instance if an error happened, and the rest of the arguments will be lled with one argument from ea of the callbas. Lets see a qui example. Imagine that one asynronous function has to be called 10 times in parallel, and that, when all done, a callba function will be called. Here is the asynronous function:
1 2 3 4

var async = function(order, callback) { var timeout = Math.round(Math.random() * 1000); setTimeout(function() { callback(null, order); }, timeout); };

And here is a piece of code launing 10 calls to async, and collecting the results on the callba:
1 2 3 4

var Step = require('step'); Step ( function doAll() {

Callba ow

109

5 6 7 8 9 10 11 12 13

for (var i = 0; i < 10; i ++) { async(i, this.parallel()); } }, function finalize(err) { console.log('done. arguments:'); console.log(arguments); } );

As you can see, on line 6 we are using this.parallel instead of this. When all of the asynronous calls are nalized, the next callba named nalize is called, printing out the arguments, whi will be:
1 2 3 4 5 6 7 8 9 10 11

{ '0': '1': '2': '3': '4': '5': '6': '7': '8': '9': '10':

undefined, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }

e rst argument is the error, and is undened since no error occurred. e remaining arguments are one per parallel call, collecting the results in order of the call.

Tip: if the asynronous function calls the callba with more than two arguments (one error and one argument), these extra arguments will be discarded. Should you have to pass more than one object into the callba, you can pa them inside an array.

Appendix - Exercise Results

Chapter: Buers
Exercise 1
Create an uninitialized buer with 100 bytes length and ll it with bytes with values starting from 0 to 99. And then print its contents.

One solution:
Source in exercises/buer/1.js
1 2 3 4 5

var buffer = new Buffer(100); for(var i = 0; i < buffer.length; i ++) { buffer[i] = i; } console.log(buffer);

Exercise 2
Do what is asked on the previous exercise and then slice the buer with bytes ranging 40 to 60. And then print it.

One solution:
Source in: exercises/buer/2.js
1 2 3 4 5 6 7 8 9 10

var buffer = new Buffer(100); for(var i = 0; i < buffer.length; i ++) { buffer[i] = i; } console.log(buffer); var buffer2 = buffer.slice(40, 60); console.log(buffer2);

110

Appendix - Exercise Results

111

Exercise 3
Do what is asked on exercise 1 and then copy bytes ranging 40 to 60 into a new buer. And then print it.

One Solution:
Source in: exercises/buer/3.js
1 2 3 4 5 6 7 8 9 10 11

var buffer = new Buffer(100); for(var i = 0; i < buffer.length; i ++) { buffer[i] = i; } console.log(buffer); var buffer2 = new Buffer(20); buffer.copy(buffer2, 0, 40, 60); console.log(buffer2);

Chapter: Event Emitter

Exercise 1
Build a pseudo-class named Tier that emits a ti event every 1 second.

One solution:
Source in: exercises/event_emier/1.js
1 2 3 4 5 6 7 8 9

var util = require('util'), EventEmitter = require('events').EventEmitter; var Ticker = function() { var self = this; setInterval(function() { self.emit('tick'); }, 1000); };

Appendix - Exercise Results

112

Exercise 2
Build a script that instantiates one Tier and bind to the ti event, printing TICK every time it gets one.

One solution:
Source in: exercises/event_emier/2.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

var util = require('util'), EventEmitter = require('events').EventEmitter; var Ticker = function() { var self = this; setInterval(function() { self.emit('tick'); }, 1000); }; util.inherits(Ticker, EventEmitter) var ticker = new Ticker(); ticker.on('tick', function() { console.log('TICK'); });

Chapter: Low-level File System

Exercise 1 - get the size of a le
Having a le named a.txt, print the size of that les in bytes.

One solution
Source code in: exercises/fs/1.1.js
1 2 3 4 5 6

var fs = require('fs'); fs.stat(__dirname + '/a.txt', function(err, stats) { if (err) { throw err; } console.log(stats.size); });

Appendix - Exercise Results

113

Exercise 2 - read a chunk from a le

Having a le named a.txt, print bytes 10 to 14.

One solution
Source code in: exercises/fs/1.2.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

var fs = require('fs'); fs.open(__dirname + '/a.txt', 'r', function(err, fd) { if (err) { throw err; } var buffer = Buffer(5); var readBytes = 0; (function readIt() { fs.read(fd, buffer, readBytes, buffer.length - readBytes, 10 + readBytes, function(err, bytesRead) { if (err) { throw err; } readBytes += bytesRead; if (readBytes === buffer.length) { console.log(buffer); } else { readIt(); } }); })(); });

Exercise 3 - read two chunks from a le

Having a le named a.txt, print bytes 5 to 9, and when done, read bytes 10 to 14.

One solution
Source code in: exercises/fs/1.3.js

Appendix - Exercise Results

114

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

var fs = require('fs'); fs.open(__dirname + '/a.txt', 'r', function(err, fd) { if (err) { throw err; } function readSome(startingAt, byteCount, callback) { var buffer = Buffer(byteCount); var readBytes = 0; (function readIt() { fs.read(fd, buffer, readBytes, buffer.length - readBytes, startingAt + readBytes, function(err, bytesRead) { if (err) { throw err; } readBytes += bytesRead; if (readBytes === buffer.length) { callback(buffer) } else { readIt(); } }); })(); } readSome(5, 4, function(buffer1) { console.log(buffer1); readSome(10, 4, function(buffer2) { console.log(buffer2); }); }) });

Exercise 4 - Overwrite a le
Having a le named a.txt, Overwrite it with the UTF8-encoded string ABCDEFGHIJLKLMNOPQRSTUVXYZ0123456789abcdefghijklmnopqrstuvxyz.

One solution:
Source code in: exercises/fs/1.4.js
1 2

var fs = require('fs');

Appendix - Exercise Results

115

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

fs.open(__dirname + '/a.txt', 'w', function(err, fd) { if (err) { throw err; } var buffer = new Buffer('ABCDEFGHIJLKLMNOPQRSTUVXYZ0123456789' + 'abcdefghijklmnopqrstuvxyz'); var written = 0; (function writeIt() { fs.write(fd, buffer, 0 + written, buffer.length - written, 0 + written, function(err, bytesWritten) { if (err) { throw err; } written += bytesWritten; if (written === buffer.length) { console.log('done'); } else { writeIt(); } }); })(); });

Exercise 5 - append to a le
Having a le named a.txt, append utf8-encoded string abc to le a.txt.

One Solution:
Source code in: exercises/fs/1.5.js
1 2 3 4 5 6 7 8 9 10 11 12

var fs = require('fs'); fs.open(__dirname + '/a.txt', 'a', function(err, fd) { if (err) { throw err; } var buffer = new Buffer('abc'); var written = 0; (function writeIt() { fs.write(fd, buffer, 0 + written, buffer.length - written, null, function(err, bytesWritten) { if (err) { throw err; } written += bytesWritten;

Appendix - Exercise Results

116

13 14 15 16 17 18 19 20

if (written === buffer.length) { console.log('done'); } else { writeIt(); } }); })(); });

Exercise 6 - change the content of a le

Having a le named a.txt, ange byte at pos 10 to utf8 value of 7.

One solution:
Source code in: exercises/fs/1.6.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

var fs = require('fs'); fs.open(__dirname + '/a.txt', 'a', function(err, fd) { if (err) { throw err; } var buffer = new Buffer('7'); var written = 0; (function writeIt() { fs.write(fd, buffer, 0 + written, buffer.length - written, 10, function(err, bytesWritten) { if (err) { throw err; } written += bytesWritten; if (written === buffer.length) { console.log('done'); } else { writeIt(); } }); })(); });

Appendix - Exercise Results

117

Chapter: HTTP
Exercise 1
Make an HTTP server that serves les. e le path is provided in the URL like this: hp://localhost:4000/path/to/my/le.txt

One solution:
Source code in: exercises/hp/exercise_1.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

var path = require('path'), fs = require('fs'); require('http').createServer(function(req, res) { var file = path.normalize(req.url); path.exists(file, function(exists) { if (exists) { fs.stat(file, function(err, stat) { var rs; if (err) { throw err; } if (stat.isDirectory()) { res.writeHead(403); res.end('Forbidden'); } else { rs = fs.createReadStream(file); res.writeHead(200); rs.pipe(res); } }); } else { res.writeHead(404); res.end('Not found'); } }) }).listen(4000);

Appendix - Exercise Results

118

Exercise 2
Make an HTTP server that outputs plain text with 100 timestamps new-line separated every second.

One solution:
Source code in: exercises/hp/exercise_2.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

require('http').createServer(function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); var left = 10; var interval = setInterval(function() { for(var i = 0; i res.write(Date.now() + ""); } if (-- left === 0) { clearInterval(interval); res.end(); } }, 1000); }).listen(4001);

Exercise 3
Make an HTTP server that saves the request body into a le.

One solution:
Source code in: exercises/hp/exercise_3.js
1 2 3 4 5 6

var fs = require('fs'); var sequence = 0; require('http').createServer(function(req, res) { var fileName = '/tmp/' + sequence + '.bin'; console.log("writing " + fileName);

Appendix - Exercise Results

119
= fs.createWriteStream(fileName);

7 8 9 10 11 12 13 14 15

var writeStream

req.pipe(writeStream); req.on('end', function() { res.writeHead(200); res.end(); }); sequence ++; }).listen(3000);

Here we are creating a write stream (line number 7) every time there is a new request. Ea le will have a sequential le name n.bin (1.bin, 2.bin, etc.), saved to /tmp. Aer creating the write stream we pipe the request data into it (line 9). From line 10 to line 13 we are responding aer the request is done. You can test this by using curl from the command line and piping in a le like this:
1

$ curl http://localhost:3000 -T /var/log/mail.log

Exercise 4
Make a script that accepts a le name as rst command line argument and uploads this le into the server built on the previous exercise.

One solution:
Source code in: exercises/hp/exercise_4.js
1 2 3 4 5 6 7 8 9 10 11

var http = require('http'), fs = require('fs'); if (process.argv.length < 5) { console.log('Usage: ' + process.argv[0] + ' ' + process.argv[1] + ' '); return; } var options = { host: process.argv[2],

Appendix - Exercise Results

120

12 13 14 15 16 17 18 19 20

port: parseInt(process.argv[3], 10), path: '/', method: 'PUT' }; var req = http.request(options); console.log('piping ' + process.argv[4]); fs.createReadStream(process.argv[4]).pipe(req);

To test this you can try:

1

$ node exercise_4.js localhost 3000 /var/log/mail.log

Here we are initializing an HTTP put request on line 16, with the host name and ports passed in as command line. en, on line 19 we are creating a read stream from the le name and piping it to the request object. When the le read stream is nished it will call end() on the request object.

Chapter: Child processes

Exercise 1
Create a server that a) opens a le b) listens on a unix domain soet and c) spawns a client. is client opens the soet to the server and waits for a le descriptor. e server then passes in the le descriptor we opened on a). e client writes to the le and quits. When the client process quits, the server quits.

One solution:
e server code: Source code in: exercises/ild_processes/exercise_1/server.js
1 2 3 4 5 6 7

var spawn = require('child_process').spawn; require('fs').open(__dirname + '/example.txt', 'a', function(err, fileDesc) { var server = require('net').createServer(function(socket) {

Appendix - Exercise Results

121

8 9 10 11 12 13 14 15 16 17 18 19 20 21

socket.write('Here you go', fileDesc); socket.end(); server.close(); }); server.listen('/tmp/ho_child_exercise_1.sock', function() { var child = spawn(process.argv[0], [__dirname + '/client.js']); child.on('exit', function() { console.log('child exited'); }); }); });

First we open the le on line 3. Once it is opened, we create the server (line 5) and bind it to a well-known soet path (line 12). When we start listening, we spawn the ild process (line 14), whi is a node process executing ild.js from the current directory. is server will simply write the le descriptor into the rst connecting client, end the connection and close the server soet. e client code: Source code in exercises/ild_processes/exercise_1/client.js
1 2 3 4 5 6 7 8 9

var fs = require('fs'); var conn = require('net'). createConnection('/tmp/ho_child_exercise_1.sock'); conn.on('fd', function(fileDesc) { fs.write(fileDesc, "this is the child!", function() { conn.end(); }); });

e client is very simple: it will connect to the server and wait for a le descriptor to be handed down (line 4). When that happens, it will append the string this is the ild! into the le and end the server connection, exiting the process.

Appendix - Exercise Results

122

Chapter: Streaming HTTP Chunked responses

Exercise 1
Create a mixed TCP and HTTP server that, for every HTTP request, streams all the TCP clients input into the request response.

One solution:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

var util = require('util'), EventEmitter = require('events').EventEmitter; var Hose = function() { var self = this; require('net').createServer(function(socket) { socket.on('data', function(data) { self.emit('data', data); }) }).listen(4001); }; util.inherits(Hose, EventEmitter); var hoser = new Hose(); require('http').createServer(function(req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); hoser.on('data', function(data) { res.write(data); }); }).listen(4002);

Here we are creating a pseudo-class named Hose that inherits from EventEmier. An instance of this class (hoser ), when created, starts the TCP server, and on every message it emits a data event. en, the HTTP server simply binds to that event on the hoser, and when the hoser emits it, that data is piped into the response.

Appendix - Exercise Results

123

Chapter: UDP
Exercise 1
Create a UDP server that eoes the messages ot receives ba into the origin soet.

One solution:
1 2 3 4 5 6 7 8

var dgram = require('dgram'); var socket = dgram.createSocket('udp4', function(message, rinfo) { console.log(rinfo); socket.send(message, 0, message.length, rinfo.port, rinfo.address); }); socket.bind(4001);

You can test this using netcat like this:

1

$ netcat -u -w 1 localhost 4001

and then you can type a message and hit Return, and you should get the same message ba.

Chapter: TCP
Exercise 1
Make a at server that requires no authentication, just a TCP client connection. Ea time the client sends some text, the server broadcasts it to the other clients.

One solution:
Source code in: exercises/hp/exercise_1.js
1 2 3 4 5 6

var sockets = []; require('net').createServer(function(socket) { sockets.push(socket);

Appendix - Exercise Results

124

7 8 9 10 11 12 13 14 15 16 17 18 19 20

socket.on('data', function(data) { sockets.forEach(function(socket) { socket.write(data); }); }); socket.on('end', function() { var pos = sockets.indexOf(socket); if (pos > 0) { sockets.splice(pos, 1); } }); }).listen(4001);

On line 5 we are adding every new connection to the soets array. On line 8 and 9 we are broadcasting every message received to every client connection. On lines 14-17 we are removing the client soet from the soets array if he disconnects.

Exercise 2
Make a at client that accepts 2 command line arguments: host and port, and reads from stdin, sending data to the server on ea new line.

One solution:
Source code in exercises/hp/exercise_2.js
1 2 3 4 5 6 7 8 9 10 11

var net = require('net'); if (process.argv.length < 4) { console.log('Usage: ' + process.argv[0] + ' ' + process.argv[1] + ' return; } var host = process.argv[2], port = process.argv[3]; var conn = net.createConnection(port, host);

');

Appendix - Exercise Results

125

12 13 14 15

process.stdin.resume(); process.stdin.pipe(conn); conn.pipe(process.stdout, {end: false});

Here we are opening a connection to the at server on line 11. en, we pipe the process stdin into the soet (line 13). Also, to print what we get from the server, we pipe that soet into the process stdout (line 15).

Chapter: SSL / TLS

Exercise 1
Create a certicate authority. Create a client certicate signed by this new certicate authority.

One solution:
Create the Certicate Authority (CA):
1 2 3

$ mkdir private $ openssl req -new -x509 -days 3650 -extensions v3_ca -keyout private/cake\ y.pem -out cacert.pem

Here is an example output:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Generating a 1024 bit RSA private key ..............................................++++++ ..............++++++ writing new private key to 'private/cakey.pem' Enter PEM pass phrase: Verifying - Enter PEM pass phrase: ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN\ . There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. -----

Appendix - Exercise Results

126

16 17 18 19 20 21 22

Country Name (2 letter code) [AU]:PT State or Province Name (full name) [Some-State]: Locality Name (eg, city) []:Lisbon Organization Name (eg, company) [Internet Widgits Pty Ltd]:Test CA Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:Pedro Teixeira Email Address []:[email protected]

e CA should now be generated:

1 2 3 4 5 6 7 8 9

$ tree . -- cacert.pem `-- private `-- cakey.pem

Now we create the client private and public keys:

1 2 3 4 5

$ mkdir client1 $ cd client1 $ openssl genrsa -out client.pem 1024

Now we generate a Certicate Signing Request (CSR):

1

openssl req -new -key client.pem -out client_csr.pem

e output should be something like this:

1 2 3 4 5 6 7

You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN\ . There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank.

Appendix - Exercise Results

127

8 9 10 11 12 13 14 15 16 17 18 19 20

----Country Name (2 letter code) [AU]:PT State or Province Name (full name) [Some-State]: Locality Name (eg, city) []:Lisbon Organization Name (eg, company) [Internet Widgits Pty Ltd]:Test Client Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:Pedro Teixeira Email Address []:[email protected] Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:

Now we create a signed certicate:

1 2

$ openssl x509 -req -in client_csr.pem -signkey ../private/cakey.pem -out \ client_cert.pem

You will be prompted for the CA private key password, and he output will look something like this:
1 2 3 4 5

Signature ok subject=/C=PT/ST=Some-State/L=Lisbon/O=Test Client/CN=Pedro Teixeira/email\ [email protected] Getting Private key Enter pass phrase for ../private/cakey.pem:

Now you should have the client certicate in the le named client_cert.pem.

Exercise 2
Create a TLS eo server that uses the default certicate authorities.

One solution:
Create a directory to store the keys named exercise2 :
1 2 3

$ mkdir exercise2 $ cd exercise 2

Appendix - Exercise Results

128

Create the server private key:

1

$ openssl genrsa -out server-key.pem 1024

Create the server certicate sigining request (CSR):

1

$ openssl req -new -key server-key.pem -out server-csr.pem

Create a self-signed serticate:

1 2

$ openssl x509 -req -in server-csr.pem -signkey server-key.pem -out server\ -cert.pem

Now go down a dir and type the server script:

1

$ cd ..

Source code in: exercise/ssl_tls/exercise_2.js

1 2 3 4 5 6 7 8 9

var fs = require('fs'); var options = { key: fs.readFileSync(__dirname + '/exercise_2/server-key.pem'), cert: fs.readFileSync(__dirname + '/exercise_2/server-cert.pem'), ca: fs.readFileSync(__dirname + '/exercise_1/private/cakey.pem') }; require('tls').createServer(options, function(socket) { socket.pipe(socket); }).listen(4001);

Exercise 3
Create a TLS client that reads from stdin and sends it to the eo TLS server created on exercise 2.

Appendix - Exercise Results

129

One solution:
Source code in: exercise/ssl_tls/exercise_3.js
1 2 3 4 5 6 7 8

var fs = require('fs'); var client = require('tls').connect(4001, function(err) { client.connected = true; console.log('connected'); process.stdin.resume(); process.stdin.pipe(client); client.pipe(process.stdout, {end: false}); });

Exercise 4
Make the TLS server only accept connections if the client is certied. Verify that he does not let the client created on exercise 3 connect.

One solution:
Source code in: exercise/ssl_tls/exercise_4.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14

var fs =require('fs'); var options = { key: fs.readFileSync(__dirname + '/exercise_2/server-key.pem'), cert: fs.readFileSync(__dirname + '/exercise_2/server-cert.pem'), ca: fs.readFileSync(__dirname + '/exercise_1/private/cakey.pem'), requestCert: true, rejectUnauthorized: true }; require('tls').createServer(options, function(socket) { socket.on('data', function(data) { console.log(data.toString()); }); socket.pipe(socket); }).listen(4001);

Here we are passing in the requestCert and rejectUnauthorized options with a true value. Once our client connects the connection will be rejected since it is not using a certicate regognizable to the server.

Appendix - Exercise Results

130

Exercise 5
Make the TLS Server use the same certicate authority you used to sign the client certicate with. Verify that the server now accepts connections from this client.

One solution:
Source code in: exercise/ssl_tls/exercise_5.js
1 2 3 4 5 6 7 8 9 10 11 12

var fs = require('fs'); var options = { key: fs.readFileSync(__dirname + '/exercise_1/client1/client.pem'), cert: fs.readFileSync(__dirname + '/exercise_1/client1/client_cert.pem') }; var client = require('tls').connect(4001, options, function(err) { client.connected = true; console.log('connected'); process.stdin.resume(); process.stdin.pipe(client); client.pipe(process.stdout, {end: false}); });

Here we are using the client key and certicate we generated on exercise 1, whi should be recognizable by our server since it was signed by our Certicate Authority.