Visit the Lady Market in Kowloon at Noon when it is just opening
Some stalls do not get set up until later in the morning. Once the framework is set up then the vendor will put up a few items to show their goods and perhaps go off for a late breakfast! Most stalls are open until late in the evening so they don't feel bad about not starting first thing in the morning. The opening hours of the market vary but typically the market is ready for business by mid-day and not closed until 11PM.
When Visiting the Market it is actually quite good to go about Noon when things are first setting up and it is not too busy. If you browse for half an hour, then have a break for a snack or a light lunch and then have another browse while things are a little busier.
In the mid or late afternoon it will become quite busy, particularly on weekends when of course the market is much busier.