DataFlex
DataFlex
Google Drive - Build a Local Cache of Metadata
See more Google Drive Examples
This example demonstrates how to download the metadata for all files in a Google Drive account to create a local filesystem cache with the information. The cache can be used to fetch information without having to query Google Drive.Chilkat DataFlex Downloads
Use ChilkatAx-win32.pkg
Procedure Test
Boolean iSuccess
Variant vGAuth
Handle hoGAuth
Handle hoRest
Boolean iBAutoReconnect
Handle hoGdCache
Integer iNumCacheFilesDeleted
Variant vDtExpire
Handle hoDtExpire
String sAllFields
Master Handle hoJsonMaster
Variant vJsonMasterArr
Handle hoJsonMasterArr
Variant vJsonMasterPaths
Handle hoJsonMasterPaths
Handle hoJson
Variant vJsonFileMetadata
Handle hoJsonFileMetadata
Integer i
Integer iNumFiles
String sJsonResponse
Integer iPageNumber
String sPageToken
Boolean iBContinueLoop
Boolean iBHasMorePages
Handle hoHashTable
ForFileId Handle hoSbPathForFileId
Handle hoSaFileInfo
String sFileId
Handle hoSbPath
Boolean iBFinished
String sStrJsonMaster
String sTemp1
String sTemp2
Integer iTemp1
Boolean bTemp1
Move False To iSuccess
Move True To iSuccess
// It requires the Chilkat API to have been previously unlocked.
// See Global Unlock Sample for sample code.
// This example uses a previously obtained access token having permission for the
// Google Drive scope.
Get Create (RefClass(cComChilkatAuthGoogle)) To hoGAuth
If (Not(IsComObjectCreated(hoGAuth))) Begin
Send CreateComObject of hoGAuth
End
Set ComAccessToken Of hoGAuth To "GOOGLE_DRIVE_ACCESS_TOKEN"
Get Create (RefClass(cComChilkatRest)) To hoRest
If (Not(IsComObjectCreated(hoRest))) Begin
Send CreateComObject of hoRest
End
// Connect using TLS.
Move True To iBAutoReconnect
Get ComConnect Of hoRest "www.googleapis.com" 443 True iBAutoReconnect To iSuccess
// Provide the authentication credentials (i.e. the access token)
Get pvComObject of hoGAuth to vGAuth
Get ComSetAuthGoogle Of hoRest vGAuth To iSuccess
// -------------------------------------------------------------------
// Initialize our cache object. Indicate the location of the root cache directory, and how many cache levels are to exist.
// For small caches (level 0) all cache files are in the root directory.
// For medium caches (level 1) cache files are located in 256 sub-directories from the root.
// For large caches (level 2) cache files are located in 256x256 sub-directories two levels down from the root.
Get Create (RefClass(cComChilkatCache)) To hoGdCache
If (Not(IsComObjectCreated(hoGdCache))) Begin
Send CreateComObject of hoGdCache
End
Set ComLevel Of hoGdCache To 0
// Use a root directory that makes sense on your operating system..
Send ComAddRoot To hoGdCache "C:/ckCache/googleDrive"
// If we are re-building the cache, we can first delete the entire contents of the cache.
Get ComDeleteAll Of hoGdCache To iNumCacheFilesDeleted
// Create a date/time object with an time 7 days from the current date/time.
Get Create (RefClass(cComCkDateTime)) To hoDtExpire
If (Not(IsComObjectCreated(hoDtExpire))) Begin
Send CreateComObject of hoDtExpire
End
Get ComSetFromCurrentSystemTime Of hoDtExpire To iSuccess
Get ComAddDays Of hoDtExpire 7 To iSuccess
// Indicate that we want ALL possible fields.
// If no fields are indicated, then only the basic fields are returned.
Move "appProperties,capabilities,contentHints,createdTime,description,explicitlyTrashed,fileExtension,folderColorRgb,fullFileExtension,headRevisionId,iconLink,id,imageMediaMetadata,isAppAuthorized,kind,lastModifyingUser,md5Checksum,mimeType,modifiedByMeTime,modifiedTime,name,originalFilename,ownedByMe,owners,parents,permissions,properties,quotaBytesUsed,shared,sharedWithMeTime,sharingUser,size,spaces,starred,thumbnailLink,trashed,version,videoMediaMetadata,viewedByMe,viewedByMeTime,viewersCanCopyContent,webContentLink,webViewLink,writersCanShare" To sAllFields
// We're going to keep a master list of fileId's as we iterate over all the files in this Google Drive account.
// This master list will also be saved to the cache under the key "AllGoogleDriveFileIds".
Get Create (RefClass(cComChilkatJsonObject)) To hoJsonMaster
If (Not(IsComObjectCreated(hoJsonMaster))) Begin
Send CreateComObject of hoJsonMaster
End
Get Create (RefClass(cComChilkatJsonArray)) To hoJsonMasterArr
If (Not(IsComObjectCreated(hoJsonMasterArr))) Begin
Send CreateComObject of hoJsonMasterArr
End
Get pvComObject of hoJsonMasterArr to vJsonMasterArr
Get ComAppendArray2 Of hoJsonMaster "fileIds" vJsonMasterArr To iSuccess
// Also keep a list of file paths.
Get Create (RefClass(cComChilkatJsonArray)) To hoJsonMasterPaths
If (Not(IsComObjectCreated(hoJsonMasterPaths))) Begin
Send CreateComObject of hoJsonMasterPaths
End
Get pvComObject of hoJsonMasterPaths to vJsonMasterPaths
Get ComAppendArray2 Of hoJsonMaster "filePaths" vJsonMasterPaths To iSuccess
// The default page size is 100, with a max of 1000.
Get ComAddQueryParam Of hoRest "pageSize" "200" To iSuccess
Get Create (RefClass(cComChilkatJsonObject)) To hoJson
If (Not(IsComObjectCreated(hoJson))) Begin
Send CreateComObject of hoJson
End
// Send the request for the 1st page.
Get ComFullRequestNoBody Of hoRest "GET" "/drive/v3/files" To sJsonResponse
Move 1 To iPageNumber
Get ComLastMethodSuccess Of hoRest To bTemp1
Get ComResponseStatusCode Of hoRest To iTemp1
Move (bTemp1 And (iTemp1 = 200)) To iBContinueLoop
While (iBContinueLoop = True)
Showln "---- Page " iPageNumber " ----"
Get ComLoad Of hoJson sJsonResponse To iSuccess
Get ComSizeOfArray Of hoJson "files" To iNumFiles
Move 0 To i
While (i < iNumFiles)
// Add this file ID to the master list.
Set ComI Of hoJson To i
Get ComStringOf Of hoJson "files[i].id" To sTemp1
Get ComAddStringAt Of hoJsonMasterArr -1 sTemp1 To iSuccess
Move (i + 1) To i
Loop
// Get the next page of files.
// If the "nextPageToken" is present in the JSON response, then use it in the "pageToken" parameter
// for the next request. If no "nextPageToken" was present, then this was the last page of files.
Get ComStringOf Of hoJson "nextPageToken" To sPageToken
Move False To iBContinueLoop
Get ComLastMethodSuccess Of hoJson To iBHasMorePages
If (iBHasMorePages = True) Begin
Get ComClearAllQueryParams Of hoRest To iSuccess
Get ComAddQueryParam Of hoRest "pageSize" "200" To iSuccess
Get ComAddQueryParam Of hoRest "pageToken" sPageToken To iSuccess
Get ComFullRequestNoBody Of hoRest "GET" "/drive/v3/files" To sJsonResponse
Get ComLastMethodSuccess Of hoRest To bTemp1
Get ComResponseStatusCode Of hoRest To iTemp1
Move (bTemp1 And (iTemp1 = 200)) To iBContinueLoop
Move (iPageNumber + 1) To iPageNumber
End
Loop
// Check to see if the above loop exited with errors...
Get ComLastMethodSuccess Of hoRest To bTemp1
If (bTemp1 = False) Begin
Get ComLastErrorText Of hoRest To sTemp1
Showln sTemp1
Procedure_Return
End
// Check to see if the above loop exited with errors...
// A successful response will have a status code equal to 200.
Get ComResponseStatusCode Of hoRest To iTemp1
If (iTemp1 <> 200) Begin
Get ComResponseStatusCode Of hoRest To iTemp1
Showln "response status code = " iTemp1
Get ComResponseStatusText Of hoRest To sTemp1
Showln "response status text = " sTemp1
Get ComResponseHeader Of hoRest To sTemp1
Showln "response header: " sTemp1
Showln "response JSON: " sJsonResponse
Procedure_Return
End
// Iterate over the file IDs and download the metadata for each, saving each to the cache...
// Also, keep in-memory hash entries of the name and parent[0] so we can quickly
// build the path-->fileId cache entries. (Given that the Google Drive REST API uses
// fileIds, this gives us an easy way to lookup a fileId based on a filePath.)
Get Create (RefClass(cComChilkatHashtable)) To hoHashTable
If (Not(IsComObjectCreated(hoHashTable))) Begin
Send CreateComObject of hoHashTable
End
// Set the capacity of the hash table to something reasonable for the number of files
// to be hashed.
Get ComClearWithNewCapacity Of hoHashTable 521 To iSuccess
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbPathForFileId
If (Not(IsComObjectCreated(hoSbPathForFileId))) Begin
Send CreateComObject of hoSbPathForFileId
End
// Used for storing the file name and parents[0] in the hashTable.
Get Create (RefClass(cComCkStringArray)) To hoSaFileInfo
If (Not(IsComObjectCreated(hoSaFileInfo))) Begin
Send CreateComObject of hoSaFileInfo
End
Set ComUnique Of hoSaFileInfo To False
Get ComSizeOfArray Of hoJsonMaster "fileIds" To iNumFiles
Move 0 To i
While (i < iNumFiles)
Set ComI Of hoJsonMaster To i
Get ComStringOf Of hoJsonMaster "fileIds[i]" To sFileId
Get ComSetString Of hoSbPathForFileId "/drive/v3/files/" To iSuccess
Get ComAppend Of hoSbPathForFileId sFileId To iSuccess
Get ComClearAllQueryParams Of hoRest To iSuccess
Get ComAddQueryParam Of hoRest "fields" sAllFields To iSuccess
Get ComGetAsString Of hoSbPathForFileId To sTemp1
Get ComFullRequestNoBody Of hoRest "GET" sTemp1 To sJsonResponse
Get ComLastMethodSuccess Of hoRest To bTemp1
Get ComResponseStatusCode Of hoRest To iTemp1
If ((bTemp1 <> True) Or (iTemp1 <> 200)) Begin
// Force an exit of this loop..
Move 0 To iNumFiles
End
// Save this file's metadata to the local cache.
// The lookup key is the fileId.
Get pvComObject of hoDtExpire to vDtExpire
Get ComSaveTextDt Of hoGdCache sFileId vDtExpire "" sJsonResponse To iSuccess
// Get this file's name and parent[0], and put this information
// in our in-memory hashtable to be used below..
Get ComLoad Of hoJson sJsonResponse To iSuccess
Send ComClear To hoSaFileInfo
Get ComStringOf Of hoJson "name" To sTemp1
Get ComAppend Of hoSaFileInfo sTemp1 To iSuccess
Get ComStringOf Of hoJson "parents[0]" To sTemp1
Get ComAppend Of hoSaFileInfo sTemp1 To iSuccess
Get ComSerialize Of hoSaFileInfo To sTemp1
Get ComAddStr Of hoHashTable sFileId sTemp1 To iSuccess
Get ComStringOf Of hoJson "name" To sTemp1
Get ComStringOf Of hoJson "parents[0]" To sTemp2
Showln sTemp1 ", " sTemp2
Move (i + 1) To i
Loop
// Check to see if the above loop exited with errors...
Get ComLastMethodSuccess Of hoRest To bTemp1
If (bTemp1 = False) Begin
Get ComLastErrorText Of hoRest To sTemp1
Showln sTemp1
Procedure_Return
End
// Check to see if the above loop exited with errors...
// A successful response will have a status code equal to 200.
Get ComResponseStatusCode Of hoRest To iTemp1
If (iTemp1 <> 200) Begin
Get ComResponseStatusCode Of hoRest To iTemp1
Showln "response status code = " iTemp1
Get ComResponseStatusText Of hoRest To sTemp1
Showln "response status text = " sTemp1
Get ComResponseHeader Of hoRest To sTemp1
Showln "response header: " sTemp1
Showln "response JSON: " sJsonResponse
Procedure_Return
End
// Now that all the fileId's are in the cache, let's build the directory path
// for each fileID.
// (Technically, a fileId can have multiple parents, which means it can be in multiple directories
// at once. This is only going to build directory paths following the 0'th parent ID in the parents list.)
// The directory path for files in "My Drive" will be just the filename.
// For files in sub-directories, the path will be relative, such as "subdir1/subdir2/something.pdf"
//
Showln "---- building paths ----"
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbPath
If (Not(IsComObjectCreated(hoSbPath))) Begin
Send CreateComObject of hoSbPath
End
Get ComSizeOfArray Of hoJsonMaster "fileIds" To iNumFiles
Move 0 To i
While (i < iNumFiles)
Set ComI Of hoJsonMaster To i
Send ComClear To hoSbPath
Get ComStringOf Of hoJsonMaster "fileIds[i]" To sFileId
Move False To iBFinished
While (iBFinished = False)
Send ComClear To hoSaFileInfo
Get ComLookupStr Of hoHashTable sFileId To sTemp1
Get ComAppendSerialized Of hoSaFileInfo sTemp1 To iSuccess
// Append this file or directory name.
Get ComGetString Of hoSaFileInfo 0 To sTemp1
Get ComPrepend Of hoSbPath sTemp1 To iSuccess
// Get the parent fileId
Get ComGetString Of hoSaFileInfo 1 To sFileId
// If this fileId is not in the hashtable, then it's the fileId for "My Drive", and we are finished.
Get ComContains Of hoHashTable sFileId To bTemp1
If (bTemp1 = False) Begin
Move True To iBFinished
End
Else Begin
Get ComPrepend Of hoSbPath "/" To iSuccess
End
Loop
Get ComGetAsString Of hoSbPath To sTemp1
Showln i ": " sTemp1
// Store the filePath --> fileId mapping in our local cache.
Get ComStringOf Of hoJsonMaster "fileIds[i]" To sFileId
Get ComGetAsString Of hoSbPath To sTemp1
Get pvComObject of hoDtExpire to vDtExpire
Get ComSaveTextDt Of hoGdCache sTemp1 vDtExpire "" sFileId To iSuccess
Get ComGetAsString Of hoSbPath To sTemp1
Get ComAddStringAt Of hoJsonMasterPaths -1 sTemp1 To iSuccess
Move (i + 1) To i
Loop
// Save the master list of file IDs and file paths to the local cache.
Set ComEmitCompact Of hoJsonMaster To False
Get ComEmit Of hoJsonMaster To sStrJsonMaster
Get ComSaveTextNoExpire Of hoGdCache "AllGoogleDriveFileIds" "" sStrJsonMaster To iSuccess
Showln "JSON Master Record:"
Showln sStrJsonMaster
// The JSON Master Cache Record looks something like this:
// An application can load the JSON master record and iterate over all the files
// in Google Drive by file ID, or by path.
// {
// "fileIds": [
// "0B53Q6OSTWYolQlExSlBQT1phZXM",
// "0B53Q6OSTWYolVHRPVkxtYWFtZkk",
// "0B53Q6OSTWYolRGZEV3ZGUTZfNFk",
// "0B53Q6OSTWYolS2FXSjliMXQxSU0",
// "0B53Q6OSTWYolZUhxckMzb0dRMzg",
// "0B53Q6OSTWYolbUF6WS1Gei1oalk",
// "0B53Q6OSTWYola296ODZUSm5GYU0",
// "0B53Q6OSTWYolbTE3c3J5RHBUcHM",
// "0B53Q6OSTWYolTmhybWJSUGd5Q2c",
// "0B53Q6OSTWYolY2tPU1BnYW02T2c",
// "0B53Q6OSTWYolTTBBR2NvUE81Zzg",
// ],
// "filePaths": [
// "testFolder/abc/123/pigs.json",
// "testFolder/starfish20.jpg",
// "testFolder/penguins2.jpg",
// "testFolder/starfish.jpg",
// "testFolder/abc/123/starfish.jpg",
// "testFolder/abc/123/penguins.jpg",
// "testFolder/abc/123",
// "testFolder/abc",
// "testFolder/testHello.txt",
// "testFolder",
// "helloWorld.txt",
// ]
// }
Showln "Entire cache rebuilt..."
End_Procedure