Saturday, November 30, 2013

难怪港片不行了

今天看完了徐克的狄仁杰之通天帝国,实在是。。。差不忍睹!故事太牵强,情节发展太没逻辑。真不知道在看什么?还居然安排了一个潜规则的戏码,天后要我服侍大人,居然就是脱!!!晕倒。。。难道是演艺圈的潜规则太多,没有就不正常!虽然老狄同意出力,还是要弄个女人上保险!!!免得他不办事。 如果真办事,天后也需要弄个小三反腐!!!!

徐克的作品也沦落到这个水平了。。。缅怀我的年轻时光呀。。。看来我是真老了!

Outside world is wonderful!!!

Today just finish the study on the Require.js module concept and realize that my seeking is already implemented and my direction is right, to divide the whole website into modules (widgets) and load them when it is necessary. This idea emerged from 2 years ago for the mobile stock trading system. Although it is my first web/mobile project, my network system development experience make me more focus on the system data volume & network traffic. Because of the tight schedule, I didn't really fully implement it. Unit now, my own system, I decide to implement the dynamic loading feature, which the formal name should be Asynchronous Module Definition. After finish the JS/CSS/HTML loading based on the jQuery widget object, I happened to re-study the require.js. 

Then Shit!. That is what I need, asynchronous module loading, i18n... And all are nicely implemented, well organised.  After reading its history, I feel a little upset to find this nice tool late. 

Instead of struggling on why boss doesn't judge staff based on the performance, how could those guys be so shameless to take others credit or shifting the responsibility, these guys are who I am looking for. Suddenly recall my answer to my boss exist view question, 

Boss: "well, you can't work with peoples"
Me: "Well, how can I work with them?! like xxx incident, even black and white, they can lie!!!"

For sure, I can't work them. Because,

Outside world is so wonderful!  

Tuesday, November 26, 2013

终于通了!!!

DTStockAlert系统的模块终于都链接上可以通信了!。。。。终于看见一丝曙光!

Wednesday, November 20, 2013

Another HR Person

After the Monday lunch with the resigning HR manager from my ex-company, I wondering whether it is because of the HR person over professional that most company can't build up the environment like that taught in the management books.  Seems they lacks the basic sincerity. Maybe it is because their job, they have to deal with all kinds of peoples, you know, that is the area not always under the sun.

Most HR persons who I deal with before, not pleasant, one hand they will talk about the supportive, passion... all glory stuff; another hand, they they don't really believe it. 

Suddenly recall another HR person in my ex-company, is so "cool" and emotionless when she processes your request. But she will show you the warmest smile when talking about the company culture, working environment... 

So what happened this time? Shit, it is so fake!!!

I was touched when she insisted on farewell with me back when I was leaving this company 6 months ago. And really appreciate what she shared with me, 

  1. Listening to people is the key for teamwork & cooperation, which is lacked in this company. And it is far from easy like it sounds.
  2. Most HR persons works for their believing. The company culture must match with them, otherwise they will quit immediately. 
  3. She is strong mind and still believe she can improve the company culture. After all, she just join the company 1+ year
Woh! Wise and kind person! I like it. I even regret that I should know these persons earlier. It is such a big lost. But lucky, in the last minute, I still got chance to catch the wise.

So when recently I learn that she is leaving, I am so surprise!!!  my ex-company lose the talent again!? I can't help to invite her for lunch and hope she is all right. Then got all the following surprises. Are they same person?  

  1. Financial industry isn't her industry. She dislike this industry and can't settle on this industry. In October (after my leave 4 months), she finally make it out.
  2. This company main problem is decision process isn't clear, not professional, not like her previous companies, NTUC child care, NUS...
  3. This company is full of empathy although it may be a issue. And very strong teamwork! which may make your miss when you really get chance learn those big name company from inside, Apple, since they lack it
At beginning I was shock and can't get her points and slowly realize that I am so fool!. HR is just her job. Now I start to confirm my initial doubt on her sudden farewell invitation is true, she just worked for her boss to confirm whether my leave is because of that staff fire event and collect the comments on the company. 

But I am still inspired by "her wise" words, listening is the key for the teamwork and corporation. Like my favourite idiom, "No one can do anything to the person who keep lying.  It is just in the end, he/she can't believe others anymore". 

Such a pity job, HR is !

Thursday, October 31, 2013

Wonderful Performance Metrics Tool

When my crawler project is closed to the end. my concern on the performance is heavier. How to measure my system performance? there is no easy answer for it. So many module, so many parameters, most important is all these CAN'T affect the system performance and increase the module complexity.  

Even for the JSON parsing function, I spend almost 1 working day to come out the performance measurement and it is just for the unit testing ;( . The result is just like the following. 


stockprice (36648000 records) elapsed ms:376806.531 for 36000 avg:9.937 variance:5.652 Fastest:9.000 Slowest:244.000
[67, 9 x 16910, 10 x 13029, 11 x 4306, 12 x 868, 73, 13 x 248, 14 x 123, 15 x 49, 17 x 15, 16 x 33, 19 x 9, 18 x 8, 21 x 26, 20 x 11, 23 x 51, 22 x 47, 25 x 43, 24 x 41, 27 x 39, 26 x 38, 29 x 24, 28 x 28, 31 x 3, 30 x 18, 34, 35 x 2, 32 x 2, 33 x 6, 38, 39 x 2, 36 x 6, 37 x 3, 42 x 2, 43, 41, 50, 48, 54, 244]


But how about other functions.... I was almost frightened by the future workload. It seems my system launch day need be postpone. Until today, I find this wonderful library, Metrics, through the netty example. It is fantastic and save me huge time on the performance measurement and reporting. 

With just few lines, the following result will be automatically printed into the System console. If you need, it can easily output the result into CSV, log file, JMX, even provide the servlet to remotely pass the result as JSON. Wonderful!!! 

With all these tools, I almost got the insurance on my system quality.


 final ConsoleReporter reporter = ConsoleReporter.forRegistry(Metrics.defaultRegistry())
                                                    .convertRatesTo(TimeUnit.SECONDS)
                                                    .convertDurationsTo(TimeUnit.MILLISECONDS)
                                                    .build();
reporter.start(1, TimeUnit.MINUTES);

Timer timer = Metrics.newTimer(this.getClass(),"StockPrice Batch Parse","timer",new SlidingWindowReservoir(nMax));

timer.update(stopwatch2.stop().elapsed(TimeUnit.NANOSECONDS),TimeUnit.NANOSECONDS);



-- Timers ----------------------------------------------------------------------
test.JSONParserTest.StockPrice Batch Parse.timer
             count = 34356
         mean rate = 95.49 calls/second
     1-minute rate = 95.70 calls/second
     5-minute rate = 88.47 calls/second
    15-minute rate = 80.06 calls/second
               min = 9.32 milliseconds
               max = 244.26 milliseconds
              mean = 10.46 milliseconds
            stddev = 2.38 milliseconds
            median = 10.07 milliseconds
              75% <= 10.64 milliseconds
              95% <= 11.99 milliseconds
              98% <= 13.57 milliseconds
              99% <= 22.14 milliseconds
            99.9% <= 31.03 milliseconds 





Saturday, October 26, 2013

SCTP vs UDT

When need a better messaging protocol for my project DTCrawler. I find these 2 new implementation. After study them, especially the book Networks for Grid Applications I choose UDT

1. built upon the UDP (which is my preference) 
2. Provide the flow/congestion management, which it is necessary for the application  

The difference is like this. 

"UDT borrows the messaging and partial reliability semantics from SCTP. However, SCTP are specially designed for VoIP and telephony, but UDT targets general purpose data transfer. UDT unifies both messaging and streaming semantics in one protocol."

Saturday, October 19, 2013

Java Performance Tuning Study Memo

Wonderful blogs from http://java-performance.info/!!! List of articles http://java-performance.com/

Java type memory usage


byte, boolean1 byte
short, char2 bytes
int, float4 bytes
long, double8 bytes
Byte, Boolean16 bytes
Short, Character16 bytes
Integer, Float16 bytes
Long, Double24 bytes
EnumSetBitSet1 bit per value
EnumMap4 bytes (for value, nothing for key)
ArrayList4 bytes (but may be more if ArrayList capacity is seriously more than its size)
LinkedList24 bytes (fixed)
ArrayDeque4 to 8 bytes, 6 bytes on average
JDK collectionSizePossible Trove substitutionSize
HashMap32 * SIZE + 4 * CAPACITY bytesTHashMap8 * CAPACITY bytes
HashSet32 * SIZE + 4 * CAPACITY bytesTHashSet4 * CAPACITY bytes
LinkedHashMap40 * SIZE + 4 * CAPACITY bytesNone
LinkedHashSet32 * SIZE + 4 * CAPACITY bytesTLinkedHashSet8 * CAPACITY bytes
TreeMap, TreeSet40 * SIZE bytesNone
PriorityQueue4 * CAPACITY bytesNone
All Java objects start with 8 bytes containing service information like object class and its identity hash code (returned by System.identityHashCode method). Arrays have 4 more bytes (one int field) containing array length. It looks like all user-written (not JDK classes) have another reference to object Class. These fields are followed by all declared fields. All objects are aligned by 8 bytes boundary. All primitive fields must be aligned by their size (for example, chars should be aligned by 2 bytes boundary).Object reference (including any arrays) occupy 4 bytes. What does it mean for us? In order to get most use of available memory, all object fields must occupy N*8+4 bytes (4, 12, 20, 28 and so on). In this case 100% memory will contain useful data.

Java Boxing Type Caching

Byte, Short, LongCharacterIntegerFloat, Double
From -128 to 127From 0 to 127From -128 to java.lang.Integer.IntegerCache.high or 127, whichever is biggerNo caching

Java Performance Tips

Never use exceptions as return code replacement or for any likely to happen events (especially in not IO-bound methods!). Throwing an exception is too expensive - you may experience 100 times slowdown for simple methods.

Throwing an exception in Java is a very slow operation. Expect that throwing an exception costs you something between 100 and 1000 ticks in most cases.

Case Study