With flake graphite cast iron, it's the graphite flakes that make it brittle. Cohesion between layers of atoms in graphite is very low in comparison to iron. This means that graphite flakes are effectively microcracks. Seen in a polished section under moderate magnification, they look like cracks.
With high aspect ratio, there is high stress concentration at the tip of the flakes when a tensile stress is applied normal to the plane of the flake. Since there are large numbers flakes in the structure, a crack starting at the tip of one flake does not have to travel far before it encounters another so the total crack length increases very quickly until it reaches a critical length.