Async collection in a nutshell

So I am using async with Nodejs and think it might be useful to make myself a cheat sheet about Async Collection.

Why do I need Async Collections?

Me: So I want to do async operations on a batch of items...

Async: Say no more, bro.



So, aync collection library have 12 distinct groups of functions, each including what I call a base form and its derivatives. The 12 base functions are:
concat, detect,each, every, filter, groupBy, map, sortBy, reduce, transform, some, reject.



A little bit too much? Men's memory block is 4-7, right? Don't worry. Let's group them.

Let's do it in the form of a little questionare.


1. So you have a batch of items on which you want to run async functions. Is the function a truth test? 

A. Truth test. (Go to 2)        B. NO, OF COURSE NOT YOU ANIMAL. (Go to 5)


2. Do you just want a yes or no as result, or you want to have those objects as results?

A. Gimme boolean.(Go to 3)     B. Gimme Objects.(Go to 4)


3. So you want a simple yes or no. You wanna to know if every one of the items have passed the test? Or just if any of them have passed the test?

A. Every. (Use every.)               B. Any. (Use some)


4. You want group of items as results. Do you want the first one that has passed the test, or all? Or maybe all that failed the test?

A. First one passed.(Use detect). B. All passed.(Use filter). C.All failed. (Use reject.)


5. So you dont want truth test. But maybe you want a retained state between operation on each object?

A. NO, Seriously why would I want that.(Go to 6). B. YES. (Go to 9)


6.So you want normal stuff. Ok, do you need a grouped result of all operations or you are ok if they are seperately handled?

A. Separate, please. (Use each.)          B. No. I need all results to be handled together. (Go to 7)


7. So you want results of all items together. Does the sequence matter to you?

A. YES(Go to 8).       B. NO.(Use concat)


8. So you want results in right sequence. Do you have a criteria to group them, sort them, or original sequence of item is fine?

A. Group. (Use groupBy).            B.Sort. (Use sortBy)       C. Original sequence.(Use map)


9. So you want a retained state between operations. You can do it in series, or do it in parallel but with a key information. Which one?

A. Series. (Use reduce).        B.With key. (Use transform)




To sum up, let's use a chart.

I also made a small test program to tell the difference between each, concat and map.

And here's the result.

Oh, no. Almost forgot. What about the derivatives? So you may see a function like concatseries, what does the little affix 'series' mean you may wonder.

Here's a list of all the 'affixes'.

• series : The async operation is run in series. One at a time.
• limit : Runs a maximum number of async operations at a time. User can define this number through argument 'limit'.
• of : This is only used with 'each'. Let you supply  key as an arguement of the async ooperations.
• values: Used with 'map' function only. This function is used for objects and the async operation can have a key argument
• right :Only used with reduce. Run series of items in reverse order.

	Limit	Series
concat		✓
detect	✓	✓
every	✓	✓
filter	✓	✓
groupBy	✓	✓
reduce
reject	✓	✓
some	✓	✓
each	✓	✓
sortBy
map	✓	✓
transform

Special cases with each, map, reduce

each: eachOf, eachOfLimits. eachOfSeries

map: mapValues, mapValuesLimit, mapValuesSeries

reduce: reduceRight

Canvas vs SVG, P5 vs D3

Long story short, I've stepped into the wonderland of d3.js in my wild search for a visualization library, and accidentally introduced myself to SVG. As a beginner, the first question comes into my mind is :

Both are technologies for creating graphics in html5 context, how is SVG different from canvas and which one should I use?

So with a few click within google, I found this quite comprehensive article on MSDN. I will hereby put down a simple summary of my own understanding.

We will start by talking about the two different modes these two technologies respectively adopt.

1. Retained mode vs Immediate mode

These are two different modes adopted by graphics APIs. To put it simple, retained mode can be seen as having memory between frames. Let's say you put a circle on the first frame. The API enables you to refer to this same circle and update it in the coming frames. On the contrary, immediate mode does not remember what you've done in the previous frame. Each frame, the scene is drawn from scratch.

[I also stole these nice graphs from MDSN.]

 

2. Finally... canvas vs SVG, what really is the difference?

Well...basically this question is entirely explained by the previous section：canvas uses immediate mode, while SVG uses retained mode, that is all. I am not kidding. Every other argument can be derived from this point. 

SVG is designed in retained model, then how does it refer to shapes that have been drawn? Well, SVG is XML based. Shapes are referred to as elements, in DOM.

 'SVG is essentially to graphics what HTML is to text', 
according to MDSN's introduction on SVG.

 (What can I say, MDSN also provides best tutorials with great mottos that should be printed on posters.) 

If you are familiar with DOM, SVG will be an piece of extra cake for you. After all, just like HTML, it is essentially designed to work with DOM. The mindset is the same. You create an element(say, a circle), you add attributes to it, manipulate it, update or delete it. Also like HTML, it can be styled with CSS, and uses SMIL for animation.

On the other hand, canvas is really just...canvas, a fixed-size pixel map. It is called low-level graphics API - you manipulate at the pixel level. Granted, most browser APIs provides functions that enables you to draw primitive types easily, but keeps this principle in mind. Each time you need an update on the graph, you need to tell the program to redraw everything pixel by pixel.

3. Let's be practical... what can I do with each?

I will again conveniently stole some image from MDSN.

It's always useful to keep the performance factor in mind. To summarize, Canvas do better with large number of objects or/and small size of screen. SVG do better with large size of screen and small number of objects. This conclusion is predictable, as you know, SVG maintains references to objects, at a cost. While canvas redraws every pixel, also at a cost.

Now, forward to practical scenarios. Look at this graph below, which could work as a nice summary. Static images and high fidelity documents actually are talking about the same thing: the V in SVG, actually stands for vector, and S, is for scalable. Basically, you can zoom it or enlarge it to your heart's content and the image will still be crystal sharp, yeah!

On the other side, canvas is good for complex scenes, mathematical animations, real-time data, video manipulation and all other stuff that is best worked at PIXEL-LEVEL, as we've emphasized many times.

And another two scenarios talked in the article but not on the graph, which might as well be the most used scenarios - data representation, and games.

It is easy to see why in most cases data representation works better with SVG. SVG is worked on a object level, API takes care of all the referencing to objects, which you can directly bind to your data. 

On the contrary, canvas does not by nature have this object referencing, thus the problem. Let's say we are drawing a bar chart. Now my data is update from 15 to 20 and I need to make my bar from 15px long to 20px long. With SVG, it is quite easy since you can directly refer to that rectangle that is your bar and modify the attribute accordingly. But you don't have this luxury with canvas. 'Increase the bar length!' You ask. 'Bar? What bar are you talking about?' Canvas replies. It won't work unless you write code to take care of the referencing yourself. 

But, as we said, most cases. If you want a real-time representation where large changes occurs to a large number of objects every frame, you might as well do it pixel by pixel, that is, with canvas.

Last, games. If you have done game development, this is a no brainer. Most visuals in game development are handled in a frame by frame, pixel by pixel way, especially if you want smacking visual effects. But this is also to say, if we are doing web-based casual games with limited visual effects and limited number of objects, SVG might also be sufficient.

4. You said something about P5 and D3 in the title?

Yes, I did... Ok, I have to admit first that I don't know much about both, merely just took a peek in each. Also, P5 is more that a visualization library, granted. But P5 is good for some cool visual effects, check on this channel on youtube. From the quick peek, P5 uses canvas and has its main function, draw(), as a looping function that executes at a fixed time interval to redraw the whole canvas. Remind you of update() in Unity, isn't it?

On the other hand, D3.js, or data driven documents as its full name, is indeed a data visualization library in JS utilizing SVG. It is a good example of the difference between canvas and SVG. Also, I am looking forward to dive into the source codes of the two and to see how each is designed, well... as soon as I've got some free time. In that case, I might update my findings here.