In order to optimize the speed of upstream and downstream traffic flow in the bottleneck area of expressway, two reward functions based on unit distance velocity variation and SG convolution smoothing were proposed, and two Q-learning models of speed harmonization were established. The integrated simulation platform combining Excel-VBA, VISSIM and MATLAB was used to simulate the reward functions. The results show that, the reward function based on SG convolution smoothing can effectively relieve the stop-and-go traffic on the upstream of the bottleneck. The fluctuation amplitude of speed was reduced. The Q-Learning model of speed-coordinated can suggest the optimal real time speed according to the traffic state.