Secretory granule (SG) fusion is an intermediate step in SG biogenesis. However, the precise mechanism of this process is not completely understood. We show that Golgi-derived mast cell (MC) SGs enlarge through a mechanism that is dependent on phosphoinositide (PI) remodeling and fusion with LC3+ late endosomes (amphisomes), which serve as hubs for the fusion of multiple individual SGs. Amphisome formation is regulated by the tyrosine phosphatase PTPN9, while the subsequent SG fusion event is additionally regulated by the tetraspanin protein CD63 and by PI4K. We also demonstrate that fusion with amphisomes imparts to SGs their capacity of regulated release of exosomes. Finally, we show that conversion of PI(3,4,5)P3 to PI(4,5)P2 and the subsequent recruitment of dynamin stimulate SG fission. Our data unveil a key role for lipid-regulated interactions with the endocytic and autophagic systems in controlling the size and number of SGs and their capacity to release exosomes.