static void fileRead() throws IOException { long time = System.currentTimeMillis(); int bufSize = 10 * 1024 * 1024; byte[] bs = new byte[bufSize]; ByteBuffer byteBuf = ByteBuffer.allocate(bufSize); FileChannel channel = new RandomAccessFile(input_path, "r").getChannel(); FileWriter fw = null; for (int i = 0; channel.read(byteBuf) != -1; i++) { byteBuf.rewind(); int size = byteBuf.limit(); byteBuf.get(bs); fw = new FileWriter(String.format(output_path_format1, i)); String line = new String(bs, 0, size); fw.append(line + System.getProperty("line.separator")); fw.flush(); byteBuf.clear(); } fw.close(); time = System.currentTimeMillis() - time; System.out.println("file read time = " + time); } private static void bufferRead() throws IOException { long time = System.currentTimeMillis(); BufferedInputStream bis = new BufferedInputStream(new FileInputStream(new File(input_path))); int bufferSize = 10 * 1024 * 1024; BufferedReader in = new BufferedReader(new InputStreamReader(bis, "utf-8"), bufferSize); FileWriter fw = new FileWriter(String.format(output_path_format2, 0)); for (int i = 0; in.ready(); i++) { if (i % 100 == 0) { fw = new FileWriter(String.format(output_path_format2, i / 100)); } String line = in.readLine(); fw.append(line + System.getProperty("line.separator")); if (i % 100 == 0) { fw.flush(); } } in.close(); fw.close(); time = System.currentTimeMillis() - time; System.out.println("buffer read time = " + time); }

 

这是生成大文件的代码，修改for循环次数控制文件大小，下面生成的文件大小是2G左右

    <span style="color: #0000ff">private</span> <span style="color: #0000ff">static</span> String input_path = System.getProperty("user.dir") + File.separator + "data" + File.separator + "bigdata.txt"<span style="color: #000000">;
    </span><span style="color: #0000ff">private</span> <span style="color: #0000ff">static</span> String output_path_format1 = System.getProperty("user.dir") + File.separator + "data" + File.separator + "part_1_%s.txt"<span style="color: #000000">;
    </span><span style="color: #0000ff">private</span> <span style="color: #0000ff">static</span> String output_path_format2 = System.getProperty("user.dir") + File.separator + "data" + File.separator + "part_2_%s.txt"<span style="color: #000000">;
    </span><span style="color: #0000ff">private</span> <span style="color: #0000ff">static</span>  <span style="color: #0000ff">int</span> bufSize = 10 * 1024 * 1024<span style="color: #000000">;

     </span><span style="color: #0000ff">private</span> <span style="color: #0000ff">static</span> <span style="color: #0000ff">void</span> makeBigData() <span style="color: #0000ff">throws</span><span style="color: #000000"> IOException {
        FileWriter fw </span>= <span style="color: #0000ff">new</span><span style="color: #000000"> FileWriter(input_path);
        String line </span>= "start "<span style="color: #000000">;
        </span><span style="color: #0000ff">for</span> (<span style="color: #0000ff">int</span> i = 0; i < 20000; i++<span style="color: #000000">) {
            line </span>+=<span style="color: #000000"> i;
            fw.append(line </span>+ System.getProperty("line.separator"<span style="color: #000000">));
        }
        fw.flush();
        fw.close();
        System.out.println(</span>"end"<span style="color: #000000">);
    }</span>

 

三

因为是自己琢磨的，总感觉写的有点丑，特别是生成大文件那里，希望各位指正一番。

源码地址 https://github.com/247292980/spring-boot 。fork的比star还多什么道理啊。

 

java 读写操作大文件 BufferedReader和RandomAccessFile

标签：and   make   stream   read   for循环次数控制   大小   put   cep   out   

本文系统来源：https://www.cnblogs.com/ydymz/p/10141343.html

【说明】：本文章由站长整理发布，文章内容不代表本站观点，如文中有侵权行为，请与本站客服联系(QQ:254677821)！