Convert Pandas Dataframe to Kafka Datastream | Razor Think
Featured Articles

Convert Pandas Dataframe to Kafka Datastream

By December 14, 2020 No Comments

Convert Pandas Dataframe to Kafka Datastream

More articles in Featured Articles

Introduction

The document contains the code piece which converts an input pandas dataframe into a kafka stream. This code can be used with any custom block created in the RZT AI Platform.

Block Code

    # Creating an Output Topic and a Producer Object. # This should be written inside run method. output_topic,producer=self.data_handler.create_producer(display_name="< TopicAlias Name>") # The following lines of code should be written inside stream method. # A List containing the names of all columns of the dataframe. For e.g. columns = ["C1", "C2", "C3", "C4", "C5"] columns=[< ColumnNames>] # Iterating over the rows in Pandas Dataframe and sending it to Output Topic # Here df is the Pandas Dataframe for index, row in df.iterrows(): producer.send([row[x] for x in columns]) # Closing the Producer Object producer.close()

Leave a Reply