Giter VIP home page Giter VIP logo

-python-bat-'s Introduction

-python-BAT-

现在BAT笔试和面试非常容易出现大数据的问题。例如:对5亿个整数进行排序,并输出到文件中。有5亿个ip,找出出现次数最多的ip等等,这些问题的共同特点是数据量较大,并且常规的读入内存再处理行不通,或者即使内存足够大,采取暴力的方法行不通。 针对常见的BAT公司中的大数据面试和笔试问题,列出解决思路,并使用python来实现。 例子中的代码都是基于python2.7.10

##问题1:海量日志数据,提取出某日访问百度次数最多的那个IP。 ##问题2:寻找热门查询,300万个查询字符串中统计最热门的10个查询。 ##问题3:有一个1G大小的一个文件,里面每一行是一个词,词的大小不超过16字节,内存限制大小是1M。返回频数最高的100个词。 ##问题4:海量数据分布在100台电脑中,想个办法高效统计出这批数据的TOP10。 ##问题5:有10个文件,每个文件1G,每个文件的每一行存放的都是用户的query,每个文件的query都可能重复。要求你按照query的频度排##序。 ##问题6:给定a、b两个文件,各存放50亿个url,每个url各占64字节,内存限制是4G,让你找出a、b文件共同的url? ##问题7:在2.5亿个整数中找出不重复的整数,注,内存不足以容纳这2.5亿个整数。 ##问题8: 100w个数中找出最大的100个数。

-python-bat-'s People

Contributors

webary avatar yoghurtjia avatar zhisheng17 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.