When back filling pipe, how far above the top of the pipe must be covered before using backfill material that contains large aggregate?

When back filling pipe, how far above the top of the pipe must be covered before using backfill material that contains large aggregate?