Node.js Tooling Group Meeting 2020-09-18

Present

• Christopher Hiller (@boneskull)
• Wes Todd (@wesleytodd)
• Ben Coe (@bcoe)
• Bryan English (@bengl)
• Ruy Adorno (@ruyadorno)
• Darcy Clarke (@darcyclarke)
• Joe Sepi (@joesepi)
• Ian Sutherland (@iansu)

Recursive FS Operations Deep Dive

• rimraf lessons learned
  • we should have been better about calling out that the feature was.
  • we shuld have researched other products better.

• mkdirp lessons learned

• What other fs functions can we make recursive?
  • Already identified
    • chown
    • chmod:
      • https://github.com/nodejs/tooling/issues/59#issuecomment-623653013
• What new recursive things can we add?
  • copyTree/copyDir
• What are some considerations when making recursive fs functions?
  • Security
  • POSIX compatibility
  • Being similar to other languages
  • Naming conventions
  • Should we be using recursive: true or creating new functions?
  • Behaviour w/ links

Notes

• rimraf
  • Did not get properly labeled as experimental
  • Lots of contention around naming, where to put this functionality
  • Should have done a better review of other platforms and languages ahead of time
  • Complicated library, hard to refactor
• mkdirp
  • Much less contentious
  • Was more straightforward and not labeled experimental
• Other considerations
  • Should everything be experimental?
  • Should we vendor external libraries or copy them into core?
  • Come up with some usecases for these new features
  • Security reviews of proposed APIs (@nodejs/security, someone from OpenJS Foundation, academic researchers?)
  • Start with some kind of proposal or RFC
    • New argument parser did something like this
• There have been a few patch releases of rimraf, we should consider adding them
• New things
  • chmod
    • We did initially try this
      • Some concerns were raised about security
  • chown
  • watch
    • This does exist but is not the best implementation
      • Doesn't work on Linux
    • Vendor something like watchman?
  • Look over the entire fs API for ideas
  • Lots of prior art in POSIX
    • fs already tries to follow POSIX
  • readdir
    • What would recursive readdir look like?
  • utime?
  • mv/rename
  • copyDir/copyTree
    • This seems like a valuable addition
    • What happens with symlinks? Mirror copyFile
      • This is a potential security issue
      • How does something like bash handle this?
      • Is this an issue with rimraf
    • Do we just make this copyFile with recursive option?
      • Hopefully not
    • Can we do this in JS or will it involve libuv
      • https://github.com/libuv/libuv/issues/925
• POSIX compatibility
  • How strictly do we want to adhere to this?
  • Already broken this with rimraf
  • Feels kind of user hostile, especially to non-Unix users
  • Split this into high and low level APIs? Rust does this
    • POSIX like layer
    • Higher layer to implement things like rimraf, etc.
    • We tried this with rimraf and there was resistence
  • Deno is another good example to look at
• Action items
  • Look over existing fs APIs (@joesepi)
  • Start on an RFC for copyDir (@joesepi/@iansu)
  • Start spending more time on a roadmap for the group
  • More deep dive type meetings for developing things like RFCs
    • Do these outside the normal meeting time
    • On Friday on the off weeks?
  • Revisit becoming a chartered working group (@boneskull)
    • Tooling is maybe too broad?
    • Roadmap would help with this
    • Talk with Michael Dawson about the idea