Comments for Somethink to Chew On http://www.harlan.harris.name the blog of Harlan Harris Sat, 07 Jan 2012 01:55:14 +0000 hourly 1 http://wordpress.org/?v=3.2.1 Comment on ggplot and concepts — what’s right, and what’s wrong by chango http://www.harlan.harris.name/2010/03/ggplot-and-concepts-whats-right-and-whats-wrong/comment-page-1/#comment-192 chango Sat, 07 Jan 2012 01:55:14 +0000 http://www.harlan.harris.name/?p=47#comment-192 Interesting criticisms of ggplot2. I actually came to this page from R bloggers to see if Hadley had weighed in. He's extremely open to criticism and outside ideas (there are many developers of ggplot and they are an amazing community, he's not the only person driving it). In fact, ggplot2 is under going major work currently (under the guise of a separate layers package), see the thread in the ggplot2-dev group. http://groups.google.com/group/ggplot2-dev/browse_thread/thread/6d41a475a51025d1 Here he describes the what and the why of rewriting it. He's aware of the speed issues, and the nightmare of the proto objects ( which i was expecting to be is major drawback). The appropriate use of operators for clarity is a great point (though I believe commutativity a property of the object being operated on and not the operator itself) and I too believe that qplot should be scrapped. Alas, I do not understand the need to build a separate, new jjplot... but it could be cool. There's just so much awesome work going on on ggplot2. It's actually why I switched to R and Hadley's other packages are equally paradigm changing and save me a ton of work. When they defuddle me, the community is strongly supportive. Interesting criticisms of ggplot2. I actually came to this page from R bloggers to see if Hadley had weighed in. He’s extremely open to criticism and outside ideas (there are many developers of ggplot and they are an amazing community, he’s not the only person driving it).

In fact, ggplot2 is under going major work currently (under the guise of a separate layers package), see the thread in the ggplot2-dev group.

http://groups.google.com/group/ggplot2-dev/browse_thread/thread/6d41a475a51025d1

Here he describes the what and the why of rewriting it. He’s aware of the speed issues, and the nightmare of the proto objects ( which i was expecting to be is major drawback). The appropriate use of operators for clarity is a great point (though I believe commutativity a property of the object being operated on and not the operator itself) and I too believe that qplot should be scrapped. Alas, I do not understand the need to build a separate, new jjplot… but it could be cool. There’s just so much awesome work going on on ggplot2.

It’s actually why I switched to R and Hadley’s other packages are equally paradigm changing and save me a ton of work. When they defuddle me, the community is strongly supportive.

]]> Comment on Data Science, Moore’s Law, and Moneyball by Thought this was cool: 什么是数据科学(Data Science) « CWYAlpha http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-189 Thought this was cool: 什么是数据科学(Data Science) « CWYAlpha Mon, 28 Nov 2011 14:54:03 +0000 http://www.harlan.harris.name/?p=149#comment-189 [...] 参考资料：http://www.drewconway.com/zia/?p=2378http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/http://flowingdata.com/2009/06/04/rise-of-the-data-scientist/http://radar.oreilly.com/2010/06/what-is-data-science.html [...] [...] 参考资料：http://www.drewconway.com/zia/?p=2378http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/http://flowingdata.com/2009/06/04/rise-of-the-data-scientist/http://radar.oreilly.com/2010/06/what-is-data-science.html [...]

]]> Comment on Data Science, Moore’s Law, and Moneyball by Harlan http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-181 Harlan Wed, 16 Nov 2011 03:04:49 +0000 http://www.harlan.harris.name/?p=149#comment-181 Hi Chris. I think you're absolutely right. Statistical training alone is not adequate for addressing the value of many real-world questions. I think that substantial systems intuition, and the ability to simulate the outcome of business (or government, or whatever) policy changes that might be due to the results of your analysis, is critical. Probably the people who do this best are quantitative MBAs, who have the formal training to do this work and the ability to talk with non-technical business people, and OR people, who are likewise trained to simulate complex processes and estimate the real-world effects (in money, defects, whatever) of various changes. A stats PhD seems much less useful in application of data science than many other degrees. DBAs are valuable, of course, but they don't do any of the things I talked about above. Hi Chris. I think you’re absolutely right. Statistical training alone is not adequate for addressing the value of many real-world questions. I think that substantial systems intuition, and the ability to simulate the outcome of business (or government, or whatever) policy changes that might be due to the results of your analysis, is critical. Probably the people who do this best are quantitative MBAs, who have the formal training to do this work and the ability to talk with non-technical business people, and OR people, who are likewise trained to simulate complex processes and estimate the real-world effects (in money, defects, whatever) of various changes. A stats PhD seems much less useful in application of data science than many other degrees.

DBAs are valuable, of course, but they don’t do any of the things I talked about above.

]]> Comment on Data Science, Moore’s Law, and Moneyball by human mathematics http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-180 human mathematics Wed, 16 Nov 2011 02:50:46 +0000 http://www.harlan.harris.name/?p=149#comment-180 I'd like to hear your comments on the Quora post "What does one have to learn to become a data scientist?". I thought the post indicated a lack of definition to the field. I also read something in the NYT maybe a year or two ago which also claimed that stats PhD's are/will be a valuable labour force. Again, once I see the evidence, I'll believe it... I’d like to hear your comments on the Quora post “What does one have to learn to become a data scientist?”. I thought the post indicated a lack of definition to the field.

I also read something in the NYT maybe a year or two ago which also claimed that stats PhD’s are/will be a valuable labour force. Again, once I see the evidence, I’ll believe it…

]]> Comment on Data Science, Moore’s Law, and Moneyball by human mathematics http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-179 human mathematics Wed, 16 Nov 2011 02:48:22 +0000 http://www.harlan.harris.name/?p=149#comment-179 Meh ... I hope Hal Varian is right, but as of now it looks like NoSQL, Hadoop, and various other upgraded ETL/database stuff is where the money's at. I think statisticians face the problem that they can't communicate exactly what value they will add to the bottom line ("I will explain things that you don't understand to you ... which will accomplish ... um ... well I don't know your business well enough to say"). Whereas DB's are necessary to the basic functioning of a business. I see a parallel to Quants (finance). If you are creative and can come up with strategies, great once you're in the door. But you need to be an excellent programmer to get in the door (so they know they can extract some value from you as a servant, before you become an advisor). Meh … I hope Hal Varian is right, but as of now it looks like NoSQL, Hadoop, and various other upgraded ETL/database stuff is where the money’s at. I think statisticians face the problem that they can’t communicate exactly what value they will add to the bottom line (“I will explain things that you don’t understand to you … which will accomplish … um … well I don’t know your business well enough to say”). Whereas DB’s are necessary to the basic functioning of a business.

I see a parallel to Quants (finance). If you are creative and can come up with strategies, great once you’re in the door. But you need to be an excellent programmer to get in the door (so they know they can extract some value from you as a servant, before you become an advisor).

]]> Comment on Smartphones, MP3 players, and Bluetooth: the division of labor by Somethink to Chew On » Apple TV and cross-device user-interface integration http://www.harlan.harris.name/2010/03/smartphones-mp3-players-and-bluetooth-the-division-of-labor/comment-page-1/#comment-173 Somethink to Chew On » Apple TV and cross-device user-interface integration Sun, 06 Nov 2011 20:57:13 +0000 http://www.harlan.harris.name/?p=64#comment-173 [...] into a TV that would be actually compelling, though. And in some ways it’s the same way that I earlier blogged about for MP3 players. Cross-device user-interfaces. Here’s how it might work for a [...] [...] into a TV that would be actually compelling, though. And in some ways it’s the same way that I earlier blogged about for MP3 players. Cross-device user-interfaces. Here’s how it might work for a [...]

]]> Comment on ggplot and concepts — what’s right, and what’s wrong by LC http://www.harlan.harris.name/2010/03/ggplot-and-concepts-whats-right-and-whats-wrong/comment-page-1/#comment-170 LC Thu, 27 Oct 2011 00:55:59 +0000 http://www.harlan.harris.name/?p=47#comment-170 Very nice post. I do think that there are many appealing concepts and decisions introduced in ggplot, but I get burned every time I learn it because it's restrictive in the types of plots you can make with its native syntax (HW deliberately makes it difficult if not impossible to make types of plots which he disapproves, and these mostly include issues regarding axis addition or customizations of limits), and also because of the (lack of) speed. And you're quite right about the "abuse" of the operator! I couldn't quite put my finger on it but you are spot on in that it does break your intuitive understanding of what is advertised as a superposition of layers. Having complained about all this, HW's packages (reshape, plyr) are still the main reason why I use R, but in the end I still go back to lattice graphics for the plotting. It's warty, but fast and amenable for further customization than ggplot2 (and there's even a ggplot2like() themed settings available in latticeExtra). Very nice post. I do think that there are many appealing concepts and decisions introduced in ggplot, but I get burned every time I learn it because it’s restrictive in the types of plots you can make with its native syntax (HW deliberately makes it difficult if not impossible to make types of plots which he disapproves, and these mostly include issues regarding axis addition or customizations of limits), and also because of the (lack of) speed. And you’re quite right about the “abuse” of the operator! I couldn’t quite put my finger on it but you are spot on in that it does break your intuitive understanding of what is advertised as a superposition of layers. Having complained about all this, HW’s packages (reshape, plyr) are still the main reason why I use R, but in the end I still go back to lattice graphics for the plotting. It’s warty, but fast and amenable for further customization than ggplot2 (and there’s even a ggplot2like() themed settings available in latticeExtra).

]]> Comment on Data Science, Moore’s Law, and Moneyball by getstats » Getstats – Campaigning to make Britain better with numbers and statistics http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-141 getstats » Getstats – Campaigning to make Britain better with numbers and statistics Wed, 05 Oct 2011 11:37:54 +0000 http://www.harlan.harris.name/?p=149#comment-141 [...] and are statisticians data scientists? These are just two of the immediate questions that ‘Data Science, Moore’s Law and Moneyball’ a recent piece on Harlan Harris’s blog, has raised for [...] [...] and are statisticians data scientists? These are just two of the immediate questions that ‘Data Science, Moore’s Law and Moneyball’ a recent piece on Harlan Harris’s blog, has raised for [...]

]]> Comment on Data Science, Moore’s Law, and Moneyball by Mic http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-136 Mic Thu, 29 Sep 2011 20:43:29 +0000 http://www.harlan.harris.name/?p=149#comment-136 I liked your post! I'm a physicist by training, but I've been working in the "data science" and "analytics" area for going on 20 years now. It's just never been called that before... It's really coming into play now because lots of data is available, computing power is cheap, and smart people can get answers from the data quickly using these tools (and creating their own). The marketplace is just now ready to recognize "data science" on its own terms. I liked your post! I’m a physicist by training, but I’ve been working in the “data science” and “analytics” area for going on 20 years now. It’s just never been called that before…

It’s really coming into play now because lots of data is available, computing power is cheap, and smart people can get answers from the data quickly using these tools (and creating their own). The marketplace is just now ready to recognize “data science” on its own terms.

]]> Comment on Data Science, Moore’s Law, and Moneyball by Taylor http://www.harlan.harris.name/2011/09/data-science-moores-law-and-moneyball/comment-page-1/#comment-131 Taylor Thu, 29 Sep 2011 14:13:33 +0000 http://www.harlan.harris.name/?p=149#comment-131 Great post, Harlan. I'd always been drawn to the idea that a data scientist is a career defined by an eclectic path or skill set, instead of one set of tools and procedures. However, I'd never seen that definition put forth until your talk. Point #2 was me, but it was a result of all the other great discussion from the DC meetup. Great post, Harlan. I’d always been drawn to the idea that a data scientist is a career defined by an eclectic path or skill set, instead of one set of tools and procedures. However, I’d never seen that definition put forth until your talk. Point #2 was me, but it was a result of all the other great discussion from the DC meetup.

]]>