Hadoop云计算外文翻译文献文档格式.docx

1、原文:Meet HadoopIn pioneer days they used oxen for heavy pulling, and when one ox couldnt budge a log, they didnt try to grow a larger ox. We shouldnt be trying for bigger computers, bumt foorre systems of computers.- 18 -Data!Grace HopperWe live in the data age. Its not easy to measure the total volu

2、me of data stored electronically,but an IDC estimate put the size of the “digital universe” at 0.18 zettabyintes2006, and is forecasting a tenfold growth by 2011 to 1.8 zettabytes. A zettabyte is 1021 bytes, or equivalently one thousand exabytes, one million petabytes, or one billitoernabytes. Thats

3、roughly the same order of magnitude as one disk drive for every person in the world.This flood of data is coming from many sources. Consider the following: The New York Stock Exchange generates about one terabyte of new trade data per day. Facebook hosts approximately 10 billion photos, taking up on

4、e petabyte of storage. A, the genealogy site, storaersound 2.5 petabytes of data. The Internet Archive stores around 2 petabytes of data, and is growing at a rate of 20 terabytes per month. The Large Hadron Collider near Geneva, Switzerland, will produce aboupt e1t5abytes ofdata per year.So theresa

5、lot of data out there. But you are probably wondering how it affects you. Most of the data is locked up in the largest web properties （like search engines）, or scientific or financial institutions, isnt it? Does the advent of “Big Data,” as it is beingcalled, affect smaller organizations or individu

6、als?I argue that it does. Take photos, for example. My wifes grandfather was an avid photographer, and took photographs throughout his adult life. His entire corpus of medium format, slide, and 35mm film, when scanned in at high-resolution, occupies around 10 gigabytes. Compare this to the digital p

7、hotos that my family took last year,which take up about 5 gigabytes of space. My family is producing photographic daatat 35 times the rate my wifes grandfathesrdid, and the rate is increasing every year as it becomes easier to take more and more photos. More generally, the digital streams that indiv

8、iduals are producing are growing apace.Microsoft Researchs MyLifeBits project gives a glimpse of archiving orfspoenal information that may become commonplace in the near future. MyLifeBits was an experiment where an individuals interactionsphone calls, emails, documents were captured electronically

9、andstored for later access. The data gathered included a photo taken every minute, which resulted in an overall data volume of one gigabyte a month. When storage costs come down enough to make it feasible to store continuous audio and video, the data volume for a future MyLifeBits service will be ma

10、ny times that.The trend is for every individuals data footprint to grow, but perhaps more importatnhtely amount of data generated by machines will be even greater than that generated by people.Machine logs, RFID readers, sensor networks, vehicle GPS traces, retail transactionasll of these contribute

11、 to the growing mountain of data.The volume of data being made publicly available increases every year too. Organizations no longer have to merely manage their own data: success in the future will be dictated to a large extent by their ability to extract value from other organizations data.Initiativ

12、es such as Public Data Sets on Amazon Web Services, Infochimps.org, and theinfo.org exist to foster the “information commons,” where data can be freely （orthine case of AWS, for a modest price） shared for anyone to download and analyze. Mashups between different information sources make for unexpect

13、ed and hitherto unimaginable applications.Take, for example, the A project, which watches the Astrometry groupon Flickr for new photos of the night sky. It analyzes each image, and identifies which part of the sky it is from, and any interesting celestial bodies, such as stars or galaxAielst.hough i

14、tsstill a new and experimental service, it shows the kind of things that are possible when data （in this case, tagged photographic images） is made available andused for something （image analysis） that was not anticipated by the creator.It has been said that “More data usually beats better algorithms

15、,” which is to sayftohratsome problems （such as recommending movies or music based on past preferences）,however fiendish your algorithms are, they can often be beaten simply by having more data （and a less sophisticated algorithm）.The good news is that Big Data is here. The bad news is that we are struggling to store and analyze it.Data Storage and AnalysisThe problem is simple: while the storage capacities of hard drives have increased massively over the years, access speeds-the rate at which data can be read from drives-have not kept up. One typical drive