Because ordinal numbers (i.e., first, second, third, fourth, etc.) function as both adjectives and adverbs, the -ly adverbs firstly, secondly, thirdly, fourthly, and so on are superfluous. But despite the longstanding superstition against using them, these words are common in all types of writing, and there’s no need to avoid them. Where they become troublesome, however, is when the numbers get above fourthly or fifthly (seventhly? eleventhly? sixteenthly?).
In the first place, in the second place, etc. are unnecessarily wordy. One, two, three, etc. are even more informal than first, second, third, etc.
The slightly formal-sounding -ly adverbs appear often—for example:
Secondly, it cited strains in bank funding markets; thirdly, the economic slowdown, and its ability to cause further losses at banks, restricting their ability to give new loans . . . [Wall Street Journal]
Thirdly, it is important to understand all the features of a particular preferred issue. [Globe and Mail]
Thirdly, there is the worry that Col Gaddafi will defeat the rebellion or remain in power in Tripoli in a divided Libya. [Financial Times]
But the shorter and less formal ordinal numbers work just as well—for example:
And third, he doesn’t account for greater or lesser degrees of liberalism or conservatism. [New Republic]
Fourth, we urgently need to reform the Leaving Certificate and the CAO points system. [Irish Times]
And fifth, the easiest and most effective way to speculate on the price of oil is to leave the stuff in the ground, and there’s not a thing the American government can do about that. [The Economist]