It is a mixture of broken down rocks and tiny dead microorganisms. Over time these things are broken down so much that they are able to maintain a spongy feel.
The three main ways rocks are formed:

Sedimentary rocks are formed through the gradual accumulation of sediment: for example, sand on a beach or mud on a river bed.This process is known as lithification.
Igneous rocks are rocks which have crystallized from a melt or magma. The melt is made up of various components of pre-existing rock.