It can be very hard — if you think of some of the top games e.g. Call of Duty, Duke Nukem Forever, etc, these have required months (or maybe even years) of design and development from a team of programmers. It is easy to forget the skills required for some of these games e.g. mathematics and creative design.
Nevertheless, it is possible to make simple games with basic programming skills: have you used Scratch?