Deep shale gas (burial depth > 3500 m) in the Longmaxi Formation of southern Sichuan Province will be the primary target for exploration and development in China for a relatively long period. However, the lack of a physical basis for the “sweet-spots” seismic and well-logging prediction is caused by uncertainty in the rock physical properties of deep shale gas in the research area. Acoustic and hardness measurements were performed on shale samples from a deep layer of the Longmaxi Formation in southern Sichuan. Microtextural characteristics of the shale samples were also analyzed by conventional optical microscopy and scanning electron microscopy. Based on these measurements, the rock physical properties of the shale samples and control factors are discussed. It is shown that the deep shale samples have similar properties to the shallow shale in mineral composition, microtexture, and pore type. However, the organic pore in deep shale samples is relatively undeveloped, while the dissolved pores are more developed. For high-quality shale samples (total organic content > 2%), crystal quartz of biological origin forms the framework of rock samples, resulting in effective dynamic and static properties, reflecting the elastic behavior of rigid quartz aggregates. For organic-lean samples (total organic content < 2%), orientated detrital clay particles take the role of load-bearing grains. Therefore, these shale samples’ overall rock physical properties are mainly controlled by the elastic properties of “soft” clay. The load-bearing grain variation from organic-rich shale samples to organic-lean samples results in an overturned “V”-type change in terms of velocity versus content. Organic-rich shale samples also show an apparent low Poisson’s ratio. Organic-rich shale has a slight velocity–porosity trend, while organic-lean shale shows a significant velocity–porosity trend. In addition, due to the difference in rock microtexture between organic-rich and organic-lean shale, these two kinds of reservoir rocks can be discriminated in cross plots of P-wave impedance versus Poisson’s ratio and Young’s modulus versus Poisson’s ratio. Change in hardness also reflects the control of microtexture, and shale samples with biological-origin quartz as load-bearing grains show higher hardness and brittleness. However, the variation in quartz content has less of an impact on hardness and brittleness in shale samples with clay as the load-bearing grain. Our results provide an experimental basis for the geophysical identification and prediction of deep shale gas layers.