Skip to main content

Zyte automatic extraction to Azure Blob CSV file

Zyte automatic extraction items using them to create CSV file entries in Azure Blob Container

Share

Zyte automatic extraction items

Used integrations:
class AxiosSourceZyteAutomaticExtraction {    async init() {        // TODO: Create your axios credential with zyte information:        // More info at https://docs.yepcode.io/integrations/axios/#credential-configuration        // Get your Zyte API key following guide at: https://docs.zyte.com/automatic-extraction-get-started.html        // baseUrl: https://autoextract.scrapinghub.com/v1        // user: your_automatic_extraction_api_key        // password: empty        // HTTP Headers: {        //    "Content-Type": "application/json",        // }        this.axiosClient = yepcode.integration.axios(            "your-axios-zyte-credential-name"        );    }    async fetch(publish, done) {        // TODO: Customize your request checking the API documentation:        // https://docs.zyte.com/automatic-extraction.html        const payload = [            // In this sample we'll use the product list extraction: https://docs.zyte.com/automatic-extraction/product-list.html            // TODO: Other lists or single object extractions may be done changing this payload:            //       articles, comments, job posting, real state, reviews            {                url: "http://books.toscrape.com/",                pageType: "productList",            },        ];        const {            data        } = await this.axiosClient.post(`/extract`, payload);        if (data && data[0]) {            // TODO: If retrieved list is not a productList, this attribute navigations should be changed            // TODO: You may also keep on retrieving next paginations using information at data[0].productList.paginationNext            for (const item of data[0].productList.products) {                await publish(item);            }        }        done();    }    async close() {}}

Create CSV file entries in Azure Blob Container

Used integrations:
class AzureBlobTargetUploadCsv {    async init() {        // TODO: Create your azure blob credential        // More info at https://docs.yepcode.io/integrations/azure-blob/#credential-configuration        this.azureBlob = yepcode.integration.azureBlob(            "your-azure-blob-credential-name"        );        // TODO: customize the container name and blob name to upload        // // More info at https://docs.microsoft.com/en-us/javascript/api/@azure/storage-blob/blobserviceclient?view=azure-node-latest#@azure-storage-blob-blobserviceclient-getcontainerclient        const containerClient = this.azureBlob.getContainerClient("container-name");        const blockBlobClient = containerClient.getBlockBlobClient("blob-name");        // Transforms the items into a csv format        this.stringifier = csv.stringify({            delimiter: ",",        });        this.upload = blockBlobClient.uploadStream(this.stringifier);    }    async consume(item) {        // TODO: customize the csv row to create from your item content        const csvRow = [item.value, item.text];        this.stringifier.write(csvRow);    }    async close() {        this.stringifier.end();        await this.upload;    }}

Find related recipes

  • Azure Blob
  • CSV
  • Zyte

FAQs

These recipes are a good starting point for you to build your own YepCode processes and solve your integration and automation problems.

YepCode is a SaaS platform that allows to create, execute and monitor integrations and automations using source code in a serverless environment.

We like to call it the Zapier for developers, since we bring all the agility and benefits of NoCode tools (avoid server provisioning, environment configuration, deployments,...), but with all the power of being able to use a programming language like JavaScript with a NodeJS engine.

You only have to fill the sign up form and your account will be created with our FREE plan (no credit card required).

YepCode has been created with a clear enterprise approach (multi-tenant environment, team management, high security and auditing standards, IdP integrations, on-premise options,...) so we can be the Swiss army knife of any team of engineering, especially those that need to extract or send information to external systems, and where a certain dynamism or adaptation to change is necessary in that process.

Sure! You just need to do some configuration to allow YepCode servers to connect to that service. Check our docs page to get more information.

Ask us for help to solve your integration problem

Recipes may be used in a self service approach cloning them to one YepCode account, but if you are looking for a turnkey approach, our team may help you.