Bring coconut milk to a very gentle simmer over low heat. Add chocolate and agave, and stir with a whisk or molinillo until chocolate has dissolved. Pour into a container and let it cool completely in the refrigerator or freezer.

Put chocolate mixture, avocado, and cornstarch in the blender, and puree until smooth. Pour into Popsicle molds, snap on lids, insert Popsicle sticks, and freeze until solid, for 6 hours or overnight.