You can delete an item from a folder with VBA, but it then goes into the Deleted items folder, so you need to delete it from there also. This is easy to do if there is nothing in the deleted items folder, or if the deleted item is the latest item in the deleted folder. If not you are going to have to search for it e.g. as follows.
Code:
Sub DeleteSelectedItem()
'Graham Mayor - http://www.gmayor.com - Last updated - 01 Jan 2018
Dim olItem As Object
Dim lngDel As Long
Dim strSubject As String
Dim strDate As String
Dim objDeletedFolder As Folder
Set objDeletedFolder = Session.GetDefaultFolder(olFolderDeletedItems)
On Error Resume Next
Set olItem = ActiveExplorer.Selection.Item(1)
strSubject = olItem.Subject
strDate = olItem.CreationTime
olItem.Delete
For lngDel = objDeletedFolder.Items.Count To 1 Step -1
With objDeletedFolder.Items(lngDel)
If .Subject = strSubject And .CreationTime = strDate Then
objDeletedFolder.Items(lngDel).Delete
Exit For
End If
End With
Next lngDel
lbl_Exit:
Set objDeletedFolder = Nothing
Set olItem = Nothing
Exit Sub
End Sub