Supervised learning requires labeled data to train models and then make predictions from new input data. Deep Learning (DL) methods require immense amounts of training data and processing power to provide reasonable results. In computer vision applications, and more specifically in despeckling SAR (Synthetic Aperture Radar) images, due to the speckle content, there is no ground truth available. To test the performances of despeckling filters, the common approach is tocorrupt synthetic images with a suitable speckle model and then, after filtering, well-known metrics are obtained. Then, filters are tested on actual SAR data, and specific metrics for SAR are evaluated. However, even the most elaborated speckle models are far from accounting for the complex mechanisms related to SAR images. In this paper, a methodology to design a realistic dataset to overcome these limitations is proposed. Actual SAR images of the same scene but acquired with the same sensor on different dates are downloaded from one of the available satellite platforms. Images are properly co-registered and averaged to get a ground truth-like reference image to objectively evaluate the performance of a despeckling method. To show the benefits of the proposed methodology, an on-the-shelf deep learning approach is used to filter the data, and compared with the standard approach using synthetic corrupted images with a speckle model. The final validation on actual SAR data not included in the training phase validates the proposed dataset. From the results shown, it is recommended to test filters on the proposed more realistic dataset and abandon the usual approach.