I had a bit of trouble learning to dive, but eventually figured it out.  Here are some notes on what worked (There are SPOILERS here!!):

Learning to Dive: 

You don't get the option to dive until you are quite far into the game.  Apparently, there is a beach on the far north part of the island that you need to visit to "unlock" diving, after which you should be able to dive at any of the diving points.

• I visited that beach in the norh (past the ruins), but actually reloaded a previous save when I hadn't visited that beach, and for some reason I stil got the diving option. In that savegame, I had only visited the ruins.
• the first time I tried diving was on the main beach (near the starter cave).  When Amy ran out on the rock at low tide, she said something about seeing something shiny in the water, and the diving option became available

Where to Dive:

On the beaches there are rocks you can run out to when the tide is low.  I have only been out on the one at the main beach, but other beaches have them too.  

How to Dive: 

I found this very tricky, but here's what seemed to work:

•  When you're under water, use the stylus to make an upside-down U shape, about the width of the sandy part at the bottom.  You should slowly move downward, if you make the right shape (this takes a bit of practice).
• The little meter at the top right shows your depth.  You need to keep diving down until you get right to the bottom of the meter
• Sharks:  If you see a shark the "air" meter on the bottom right will start to flash:  Touch the air meter with the stylus and hold it there until the shark is gone, and then you can start diving again.  If you try to swim or fight it will eat you :)

Picking up stuff at the bottom:

• when you finally get to the bottom, click on the objects with the stylus, and either pick, or dig them up
• In order to actually keep your items, you need to touch the "surface" icon at the bottom left,  and then swim back to the surface using the "U" shape that you used to dive, before your air runs out
• if you run out of air before you make it back, you will be fine, but your new items will be gone, and you will lose some health.