Distributed quantum computing, in which multiple quantum processing units (QPUs) with small to moderate numbers of qubits are interconnected to form a large-scale quantum network, is a promising approach for realizing fault-tolerant universal quantum computation which requires very large number of qubits. Cavity quantum electrodynamics (QED) systems offer a viable platform for building such distributed quantum computing systems, provided that multiple atoms can be strongly coupled to an optical cavity in an individually addressable manner and that such cavities can be interconnected with minimal photon loss. In this talk, we will present our experimental progress on several fronts: a nanofiber cavity QED system operating in the strong coupling regime with a single trapped atom[1]; the demonstration of a coupled-cavities QED system in which two nanofiber cavity QED systems are coherently connected in an all-fiber fashion[2,3]; the development of high-finesse nanofiber cavities aimed at achieving high cooperativity[4-6]; and recent advancements toward the realization of distributed quantum computing using nanofiber cavity QED systems.