Without knowing specifics, violent death has to lower the numbers, but I'll admit probably not in a large way. It's the infant deaths that really bring down the life expectancy numbers in a significant way.
In my opinion, looking at national trends isn't as useful as comparing by ethnicity within a country. Obviously, a country like the US with a very heterogeneous population would be at a disadvantage when compared to the Scandinavian countries who have a more homogeneous population. From there you could throw in the socio-economic factors, heathcare access, etc. I guess I'm just not a fan of overall trends at the gross population level compared to more specific comparisons that take other factors into consideration. So yeah, I guess this doesn't necessarily add to the conversation, it's just me editorializing.
You're right that Canada and the US likely have similar methods for comparing infant death rates.