Async 201

@madbence

Topics

  1. Streams
  2. Promises
  3. Generators

Streams

What are they?

  • Working with String and Buffer (objectMode available)
  • Processing HUGE amount of data on-the-fly
  • Pipes&Filters model

Types

  • Readable
  • Writable
  • Duplex
  • Transform

Under the hood

Event emitter

  • Observer pattern
  • .on('name', fn)
  • .emit('name'[, arg1, ...])

Event emitter leaks

Event emitter leaks

  • Technically not a leak. But it is.
  • Clean up after yourself!
  • .once(), if possible

Backpressure

  • Slow writable vs fast readable

For consumers

Flowing vs non-flowing mode

Reading

  • 'data' event (flowing mode)
  • .read(bytes) (non-flowing mode)

(Switch between modes with .pause() and .resume())

Writing

  • .write(chunk, [enc], [cb])
  • .end([chunk], [enc], [cb])

Pipes!

  • $ cat in.txt | compress | encrypt > out.txt
  • fsReadStream
      .pipe(compress)
      .pipe(encrypt)
      .pipe(fsWriteStream);

Complex example: Stream Playground

An entire build system built on pipes: Gulp

For implementors

  • ._read(bytes) for readable, .push(chunk) to the internal buffer
  • ._write(chunk, enc, cb) for writable
  • ._transform(chunk, enc, cb) for transform

Example transform stream


var f = function(chunk, enc, cb) {
   var string = chunk.toString();
   var upper = string.toUpperCase();
  this.push(upper, enc);
  cb();
};
MyTransform.prototype._transform = f;

Promises

WTF?

  • They represent the result of a pending operation (Deferred object)
  • They were part of node a while back ago
  • They are part of ES6
  • It's a standard! (Promises/A+)
  • jQuery has a broken implementation :(
  • I don't like them! :(

In a nutshell

function sleep(ms) {
  var d = Q.defer();
  setTimeout(function() {
    d.resolve();
  }, ms);
  return d.promise;
}
sleep(1337).then(function() {
 console.log('done! :3');
});

But that's not magic! :(

The Magic

Chaining!

a1().then(function(val) {
  return a2(val);
}).then(function(val) {
  return a3(val);
}).then(function(val) {
  console.log('done!');
}, function(err) {
  console.log('err!');
});
try {
  var v = a1();
  v = a2(v);
  v = a3(v);
  console.log('done!');
} catch(e) {
  console.log('err!');
}
  • Callback-Hell avoided!
  • Errors propagate
  • Libraries (Q, etc) can provide stack-trace & basic control flow (forks/join, etc)!

Too much boilerplate code :(

Generators

Since v0.11.3

with --harmony flags (or use gnode)

function* range(a, b) {
  while(a < b) {
    yield a++;
  }
}

for(let i of range(2, 8)) {
  console.log(i); //2, 3, 4, 5, 6, 7
}

Use yield to suspend execution, .next() to continue

The idea

magic(function* (resume) {
  try {
    var result = yield fs
      .readFile('./existing.file', resume);
    console.log(result);
    var result2 = yield fs
      .readFile('./missing.file', resume);
  } catch(e) {
    console.error(e);
  }
});

The magic

simplified version of genzen (my library)

function magic(f) {
  var g = f(function(err, result) {
    if(err) {
      return g.throw(err);
    }
    g.next(result);
  });
  g.next();
}

Use a robust library, like co, galaxy, suspend, etc. They work with promises, can do basic control flow, etc...

Koa

yield express();

The next version of express.js

  • generator based middlewares
  • no built-in features, DIY

Thank you!

Questions?