Dean Chen

musings on technology and economics

Read this first

Talk on Apache Spark Internals/RDDs

I gave a talk at ScalaBay on how the RDD abstraction in Apache Spark enables diverse, distributed, and big data use cases in a cohesive package. The talk starts with RDD usage in the Spark API and shows how the same RDD interface is leveraged in Spark SQL, GraphX, MLlib and Spark Streaming to create a unified ecosystem.

It was my first time giving this talk with the specifics thought up in real time so I apologize for the hesitation in my tone.

If you are interested in learning more about Spark internals check out talks by two Spark committers: Reynold Xin, and Aaron Davidson. Some of the content in my talk is lifted from Reynold’s.

Continue reading →


The Best Design Decision Apple Made in Swift

Apple announced the Swift programming language during this morning’s WWDC. It was lots of fun to read through the five hundred page documentation on iBooks. I am very impressed with how elegantly Apple incorporated functional programming concepts and am very excited about what mainstream adoption of Swift will mean for functional programming.

There is a lot to like about Swift, but my favorite is that only variables explicitly marked as nillable can be assigned a null value. Normal variables cannot be null and accesses to nillable variables have to explicitly unwrap the nil value with a ! or chained with ?.

“nil cannot be used with non-optional constants and variables. If a constant or variable in your code needs to be able to cope with the absence of a value under certain conditions, always declare it as an optional value of the appropriate type.”

Swift will be the first mainstream...

Continue reading →


Computer Science and Math

I came across an interesting discussion regarding the role of math in computer science education on SIGCSE’s (ACM Special Interest Group on Computer Science Education consisting of CS professors) mailing list. Brad Zanden, a professor in the department of electrical engineering and computer science at the University of Tennessee, started the conversation with:

I understand that discussing the role of math in CS is one of those religious war type issues… After 30 years in the field, I still fail to see how calculus and continuous math correlate with one’s ability to succeed in many areas of computer science… I have seen many outstanding programmers who struggled with calculus and never really got it. I also constantly see how what I consider to be excessive continuous math requirements in our program (calculus 1, 2, and 3, plus linear algebra) stops students from entering computer...

Continue reading →


The Fall of AMD

Note: This is an old post from 2007

AMD’s latest flagship Phenom processor is an utter disappointment and marks the beginning of the end to the once successful company that dominated the enthusiast CPU market. AMD shares are at a 4 year low of $9.67 after another quarter with losses in the hundreds of millions[400 to be specific], with debts nearing 9 BILLION dollars. Its fall was just as surprising and quick as its rise. After hitting a high $40 in Q1 of 2006, it lost half of its value in just a year, only to decrease by 50% again this year now sitting below $10.

There are many personal feelings associated with AMD since it was one of my first and most successful investments in the stock market; I purchased the stock in 2004 and sold it the next year for a nice profit. AMD’s processors were also the CPU of my choice and the choice countless other computer enthusiasts since 2003. I...

Continue reading →