Web Scraping in Excel VBA Using Internet Explorer: Complete Guide

Internet Explorer Automation (Old Method)

Kaise kaam karta hai: VBA Internet Explorer ko open karke webpage load karta hai.

1) Opening or closing browser

Object library required:-
(Tools → References → Microsoft Internet Controls)

InternetExplorer → class from Microsoft Internet Controls

Excel VBA Code

 
Sub Test()

    Dim ie As InternetExplorer

    Set ie = New InternetExplorer
    ie.Visible = True
    ie.Navigate "https://www.google.com/"   ' navigate any url

    ' ie.Quit   ' internet explorer will be close

End Sub

You should know that the following two lines of code do the same job and are commonly used by VBA developers:

Set ie = New InternetExplorer
Set ie = CreateObject("InternetExplorer.Application")

Early Binding
Dim ie As InternetExplorer
Set ie = New InternetExplorer

You are creating an Internet Explorer object using Early Binding
VBA already knows what InternetExplorer is
You MUST enable: Microsoft Internet Controls (Reference)

Late Binding
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")

You are creating IE using Late Binding
VBA does NOT know object type in advance
No reference needed

2) Control Existing/already opened browser (Internet explorer)

Excel VBA Code

 
Sub Test()

    Dim sh As SHDocVw.ShellWindows
    Dim ie As SHDocVw.InternetExplorer

    Set sh = New SHDocVw.ShellWindows

    For Each ie In sh

        ' If ie.Name = "Internet Explorer" Then
            Debug.Print ie.Name   ' You can replace ie.Name with ie.LocationURL
        ' End If

    Next ie

End Sub

2

Excel VBA Code

 
Sub test()

Dim sh As SHDocVw.ShellWindows
Dim ie As SHDocVw.InternetExplorer

Set sh = New SHDocVw.ShellWindows

For Each ie In sh
    If ie.Name = "internet Explorer" Then
        If InStr(ie.LocationURL, "Youtube") > 0 Then
            Debug.Print ie.LocationURL
        Else
            Debug.Print "Not Found"
        End If
        
    End If
Next


End Sub

3

Excel VBA Code

 
Sub test()

‘ Run for 5 second and Quit internet explorer
Dim ie As InternetExplorer
Set ie = New InternetExplorer

ie.Visible = True
ie.Navigate "https://www.google.com"


Application.Wait (Now + TimeValue("00:00:05"))


ie.Quit

End Sub

Method 1

Application.Wait (Now + TimeValue("00:00:05")) 

Loading of this url can take some time. This is hardcore method to wait. It is not good idea method to wait. We should use browser ready state or busy state.

Method 2

Do Until ie.ReadyState = READYSTATE_COMPLETE And ie.Busy = False
    DoEvents
Loop

Method 3

Do While ie.Busy Or ie.ReadyState <> 4
    DoEvents
    If Timer - t > 15 Then Exit Do   ' 15 seconds timeout
Loop

3) Object identification techniques

Object library required:-
(Tools → References → Microsoft HTML object library)

Dim Doc As HTMLDocument
Set Doc = ie.Document
Doc.getElementById("searchInput").Value = "Hello"

Excel VBA Code

 
Sub test()

Dim ie As InternetExplorer
Set ie = New InternetExplorer

ie.Visible = True
ie.Navigate "https://www.wikipedia.org/"


Application.Wait (Now + TimeValue("00:00:05"))  ' This will wait 5 sec to load url


Dim Doc As HTMLDocument
Set Doc = ie.Document

'Doc.getElementById("searchInput").Value = "Hello"
Doc.getElementsByName("search")(0).Value = "Hello"

Application.Wait (Now + TimeValue("00:00:05"))
ie.Quit

End Sub

For input box

Doc.getElementById("searchInput").Value = "Hello"
Doc.getElementsByName("search")(0).Value = "Hello"

 For combobox

Doc.getElementById("idname").selectedIndex = 3

 For Radio Button & Checkbox

Doc.getElementById("idname").click

For more than one element having same tagname eg input, div, br etc
We create new dim as object

Dim elems As Object
Dim elem As Object