Here’s a short excerpt from an interview David Lynch did with MTV in the 80s addressing the Return of the Jedi rumors:
MTV: Is it true you almost directed “Return of the Jedi”? How close did you come?
Lynch: Not close at all. I had a meeting with George [Lucas]. I like George. It was his thing. I said, “You should direct this. It’s your thing! It’s not my thing.”
MTV: Did he flat-out offer it to you at the time?
MTV: But you immediately declined.
Lynch: I called him the next day.