A public blockchain is simply that, public. Anybody who wants to write, read, or get involved with a public blockchain is capable of doing so. Public blockchains are decentralized, meaning no one entity has control over the network. They're secure in that once data is validated on the blockchain it cannot be changed.
In short, anybody, anywhere, can submit data and transactions using a public blockchain, provided that they are connected to the network. For example, bitcoin was the first public blockchain when it was developed in 2008 and proved that money could be moved worldwide without third-parties, such as money transmitters or banks.
ICO and Blockchain lawyers state that private blockchains function much like public blockchains; however, they have access controls, which restrict who can take part in the network. This means that it works like today's centralized database systems, which limit access to specific users. On the other hand, one or multiple entities control the network in a private blockchain. This will cause you to still need to depend on third-parties to transact.
Public blockchains still have lots of hard technological issues that should be solved, starting with privacy. Transactions that occur on a public blockchain are public, meaning that anybody can see them occur.
It's very much in contrast with the way traditional and society enterprises have operated. Private blockchains add privacy controls at the detriment of security to solve this. In practice, private blockchains function as a middle ground between decentralized public blockchains and traditional cloud computing.
Another difficult technological issue with public blockchain is scaling, meaning they cannot take on vast amount of transactions at present. On the other hand, private blockchain features most of the scaling capabilities we expect of our cloud computing nowadays.
To sum it up, public blockchain comes with restricted scaling and privacy problems, which prevent adoption in a mass scale. Private blockchain solves these issues by becoming a hybrid of blockchain and cloud computing rolled into one.
The debate between public vs. private blockchain is similar to that which took place during the late 80s and early 90s between the internet vs. the intranet. Back then, most people believed that there were lots of technological obstacles to accessing the internet and no reason a business would need to be externally connected. So, businesses built intranets. Ultimately, everybody had to connect to the internet since it became rather clear that intranet networks were not the future.