You can't ride your bicycle directly to East Boston unless you want to cover a lot of miles I haven't bothered to discover going through Somerville to Chelsea to Eastie. It seems to be a bit much. You can take a bicycle on the Blue Line, however, from Aquarium to anywhere further down the line under Boston Harbor. No bridge connects Eastie to Downtown for bicyclists, only the Blue Line tunnel.
You can't walk to Eastie either, except by the roundabout, multi-city route I've imagined. You have to take the Blue Line. Eastie is a very walkable neighborhood dense with homes and little businesses arranged around four- and five-point squares. A polyglot of languages are spoken and a global array of ethnic foods and fashions are on sale.
Eastie isn't large and it is walkable, surrounded with stunning harbor and skyline views. A bicycle makes it more traversable from one side to the other as long as you keep your eyes peeled for traffic hazards. It's a busy place. It's nice to bike there if you enjoy the urban environment and you want to zip between locales. There is also a linear park that leads from the waterfront to about midway through the neighborhood just past the Airport station on the Blue Line.