It provides the player extra mana that can be obtained by shooting the Shard with a gem. After the shards' mana are depleted, they can build on that spot.
In GemCraft Chapter 2, some shards can appear on the map via sparks, and dropping gem bombs on it will yield larger shard storage. Sometimes, it can have a purple shell layer, which prevents harvesting until it's broken. Another version of it exists, called the corrupted mana shard, that has infinite storage in exchange for decreased damage from gems attacking in towers.