Inkonzo
Indlela yokucoca idatha ye-Spark Streaming
(I) DStream kunye ne-RDD
Njengoko sisazi, ukubalwa kwe-Spark Streaming kusekwe kwi-Spark Core, kwaye undoqo we-Spark Core yi-RDD, ngoko ke ukusasazwa kwe-Spark kufuneka kunxulumane ne-RDD ngokunjalo.Nangona kunjalo, ukusasazwa kwe-Spark akuvumeli abasebenzisi ukuba basebenzise i-RDD ngokuthe ngqo, kodwa i-abstracts iseti yeengqikelelo ze-DStream, i-DStream kunye ne-RDD zibudlelwane obubandakanyayo, unokuyiqonda njengepateni yokuhlobisa kwiJava, oko kukuthi, i-DStream luphuculo lwe-RDD, kodwa ukuziphatha kufana ne-RDD.
I-DStream kunye ne-RDD zombini zineemeko ezininzi.
(1) zineentshukumo ezifanayo zoguqulo, ezinje ngemephu, ukunciphisaByKey, njl. njl., kodwa kwanezinye ezikhethekileyo, njenge Window, mapWithSated, njl.
(2) zonke zineentshukumo zokuSebenza, ezifana ne-foreachRDD, ukubala, njl.
Imodeli yokucwangcisa iyahambelana.
(B) Ukwaziswa kwe-DStream kwi-Spark Streaming
I-DStream iqulethe iiklasi ezininzi.
(1) Iiklasi zemithombo yedatha, njenge-InputDStream, ngokukodwa njenge-DirectKafkaInputStream, njl.
(2) Iiklasi zoguqulo, ngokuqhelekileyo iMappedDStream, ShuffledDStream
(3) iiklasi zemveliso, njengeForEachDStream
Ukusuka apha ngasentla, idatha ukusuka ekuqaleni (igalelo) ukuya ekupheleni (imveliso) yenziwa yinkqubo ye-DStream, okuthetha ukuba umsebenzisi ngokuqhelekileyo akanako ukuvelisa ngokuthe ngqo kwaye asebenzise ii-RDDs, okuthetha ukuba i-DStream inethuba kunye nesibophelelo sokuba uxanduva lomjikelo wobomi bee-RDDs.
Ngamanye amazwi, iSpark Streaming ineukucoca ngokuzenzekelayoumsebenzi.
(iii) Inkqubo yokuveliswa kweRDD kwiSpark Streaming
Ukuhamba kobomi be-RDDs kwi-Spark Streaming bunzima ngolu hlobo lulandelayo.
(1) Kwi-InputDStream, idatha efunyenweyo iguqulwa ibe yi-RDD, njenge-DirectKafkaInputStream, eyenza i-KafkaRDD.
(2) ke ngeMappedDStream kunye nolunye uguqulo lwedatha, eli xesha libizwa ngokuthe ngqo i-RDD ehambelana nendlela yemephu yokuguqulwa.
(3) Kumsebenzi weklasi yokuphuma, kuphela xa i-RDD ibonakalisiwe, unokuvumela umsebenzisi ukuba enze ukugcinwa okuhambelanayo, ezinye izibalo, kunye neminye imisebenzi.
