The Mirador Mansion and the Chung King Mansion provide Cheap places to sleep while in Hong Kong... The are both on the same block...and only ONE BLOCK from the Famous Penninsula Hotel and the respected Salisbury YMCA... Very easy to walk to from both hotels...right down Nathan Street...not far.
If you are intimidated by the high prices of hotels in Hong Kong...check out these two options... I've stayed in both...
There are many "hotels" in each of these buildings... Essentially, people have rented floors or space on floors to set up their "hotel"... These are mostly basic and cheap...some are a little seedy...but not bad...considering you are right in the middle of the Kowloon district...walking distance to the harbor and the view of the Hong Kong Island side...and the Star Ferry which is very easy to use... Most taxi drivers should know where these buildings are...if not just ask to go to the Penninsula or YMCA...then you can walk up Nathan Street for about 1/1/2 blocks...they are large buildings on the right side of the street...